Nutanix Files
Nutanix Files
TECH NOTE
Nutanix Files
Copyright
Copyright 2022 Nutanix, Inc.
Nutanix, Inc.
1740 Technology Drive, Suite 150
San Jose, CA 95110
All rights reserved. This product is protected by U.S. and international copyright
and intellectual property laws. Nutanix and the Nutanix logo are registered
trademarks of Nutanix, Inc. in the United States and/or other jurisdictions.
All other brand and product names mentioned herein are for identification
purposes only and may be trademarks of their respective holders.
Nutanix Files
Contents
1. Executive Summary................................................................................................. 5
5. Third-Party Integration...................................................................................... 69
Antivirus.......................................................................................................................................................................69
File Operations Monitoring................................................................................................................................ 72
Intelligent Backup................................................................................................................................................... 73
6. Conclusion.................................................................................................................77
About Nutanix............................................................................................................. 78
List of Figures........................................................................................................................................................................ 79
Nutanix Files
1. Executive Summary
Nutanix Files is a software-defined, scale-out file storage solution that
provides a repository for unstructured data, such as home directories, user
profiles, departmental shares, application logs, backups, and archives. Flexible
and responsive to workload requirements, Files is a fully integrated, core
component of Nutanix.
You can deploy Nutanix Files on an existing or standalone cluster. Unlike
standalone NAS appliances, Files consolidates VM and file storage, eliminating
the need to create an infrastructure silo. Administrators can manage Files with
Nutanix Prism, just like VM services, which unifies and simplifies management.
Integration with Active Directory enables support for quotas and access-based
enumeration (ABE), as well as self-service restores with the Windows Previous
Versions feature. Nutanix Files also supports native remote replication and file
server cloning, which lets you back up Files off-site and run antivirus scans and
machine learning without affecting production.
Nutanix Files can run on a dedicated cluster or be collocated on a cluster
running user VMs. Nutanix supports Files with both ESXi and AHV. Files
includes native high availability and uses AOS storage for intracluster data
resiliency. AOS storage also provides data efficiency techniques such as erasure
coding (EC-X).
Nutanix Files includes File Analytics, which gives you a variety of useful insights
into your data, including full audit trails, anomaly detection, ransomware
detection and intelligence, data age analytics, and custom reporting.
Nutanix Files can support SMB and NFS from the same FSVM, and beginning
with Files version 3.5 you can enable simultaneous SMB and NFS access to
the same share and data, commonly referred to as multiprotocol support.
Both SMB and NFS share a common library, allowing a modular approach.
InsightsDB is a NoSQL database that maintains the statistics and alerts for Files.
Zookeeper is a centralized service that maintains configuration information,
such as domain, share or export, and IP information. The Minerva NVM Service
talks to the local CVM and sends heartbeats to share health information and
help with failover.
Each FSVM stores file server data on multiple file systems that store share-
specific data. The individual file system provides the snapshot capability used
to provide Windows Previous Versions (WPV) support to clients. By using
separate file systems for each share or export, Nutanix Files can scale to
support billions of files in one cluster.
The previous diagram shows one FSVM running on a node, but you can put
multiple FSVMs on a node for multitenancy.
Networking
The FSVM has two network interfaces: the storage interface and the client
interface. The FSVM service that talks to the CVMs uses the storage interface,
which also provides access to Nutanix Volumes iSCSI vDisks in volume groups.
The storage interface helps manage deployment, failover, and maintenance
and enables control over one-click upgrades. Integration with the CVM lets the
FSVM determine if a storage fault has occurred and, if so, whether you must
take action. The FSVM service sends a heartbeat to its local CVM service each
second indicating its state.
Note: We recommend that you place the FSVM storage interface on the same network VLAN as
the Nutanix CVM iSCSI network to ensure maximum performance. By default, the iSCSI VLAN is
the management interface (eth0 of the CVM). Starting with Files 4.1, you can optionally create a
segmented iSCSI network for isolation purposes.
The client interface allows clients to connect to SMB shares and NFS exports
hosted on an FSVM. A client can connect to any FSVM client network interface
to access their file data. If a different FSVM provides the data, the client
connection automatically redirects to the correct FSVM interface. If an FSVM
fails, the client network address for the failed FSVM moves to another to
preserve data access.
Note: Client networks share the same vNIC within each FSVM. Because a vNIC is associated with
a single virtual switch, all client networks must be trunked to the physical NICs associated with
that virtual switch.
Storage
As shown in the following figure, each FSVM uses three separate vDisks: a 12
GB boot disk that contains the boot image, a 45 GB disk (/home/nutanix) that
contains logs and software state, and a 45 GB disk for Cassandra data.
Starting with the Files 3.7 release, each FSVM also has a volume group that
helps maintain auditing events and persistent handles for SMB Transparent
Failover.
A Nutanix Files cluster is a single namespace that includes a collection of file
systems used to support the SMB and NFS shares. The file systems support
dynamic metadata, which enables you to store an unlimited number of files. The
file system also supports variable block length allocations up to the default size
of 64 KB. The variable block length matches the size of the file; for example, a 1
Once this file system pool reaches 80 percent space utilization, Nutanix Files
automatically expands it to 80 TB by incorporating another four data disks
from the volume group.
This expansion process can occur two more times, increasing the file system
capacity to 120 TB with the second expansion and 140 TB with the final
expansion. Each expansion triggers when you’ve used 80 percent of the space
in the current pool.
The Nutanix Files 3.7 release introduced a scale-up function to the file system
pool. Once you’ve scaled a file system out to use all 16 disks, if that pool
reaches 80 percent capacity, it uses additional space on the existing disks in
the volume group. This vertical expansion supports a file system pool of up to
280 TB.
In summary, the maximum sizes for individual volume groups based on Nutanix
Files version are as follows:
• Nutanix Files 2.x to 3.2: 40 TB
• Nutanix Files 3.2 to 3.6: 140 TB
• Nutanix Files 3.7 forward: 280 TB
Note: You must have created the File server with a minimum version of 3.x to take advantage of
horizontal scale-out to 140 TB and vertical scale-up to 280 TB.
• SMB shares:
› Distributed (previously called home).
› Standard (previously called general).
• NFS exports:
› Distributed (previously called sharded).
› Standard (previously called nonsharded).
A standard share is an SMB share or NFS export hosted by a single FSVM. A
distributed share spreads the workload by distributing the hosting of top-level
directories across FSVMs, which also simplifies administration.
Starting with Files version 3.7, you can store files in the root of distributed NFS
shares. Distributed SMB shares don’t allow files in the root.
Distributed Shares
Distributed shares and exports distribute data by dividing the top-level
directories across all the FSVMs that make up the file server. Nutanix Files
maintains the directory mapping for each responsible FSVM using InsightsDB.
FSVMs use distributed file system (DFS) referrals for SMB and junctions for
NFSv4 to make sure that clients can connect to the right top-level directories.
Distributed share directories work well for home shares and exports because
Nutanix Files automatically spreads the workload over multiple FSVMs per
user (see the previous figure). If a user creates a share called \FileServer1\Users
that contains the top-level directories \Bob, \Becky, and \Kevin, \Bob may be on
FSVM-1, \Becky on FSVM-2, \Kevin on FSVM-3, and so on. The FSVMs use a
string hashing algorithm based on the directory names to distribute the top-
level directories.
This distribution can accommodate a large number of users or directories in a
single share or export. The scaling limits of more traditional designs can force
administrators to create multiple shares or exports in which, for example, one
set of users whose last names begin with A through M run off one controller
and users whose names begin with N through Z run off another controller. This
design limitation leads to management overhead headaches and unnecessary
Active Directory complexity. For these reasons, Nutanix Files expects to have
one SMB distributed share for the entire cluster. If you need to have more than
one home directory share, you can create additional shares as needed.
The top-level directories act as a reparse point—essentially a shortcut.
Consequently, administrators must create directories at the root of the share
for optimal load balancing. We recommend that you set permissions at the
share or export root before you deploy user folders. This step allows newly
created top-level directories to inherit permissions, so you don’t have to adjust
them after the fact using the Nutanix Files Microsoft Management Console
(MMC) plug-in.
Standard shares and exports don’t distribute top-level directories. A single file
server always owns the files and subfolders for standard shares and exports.
The following diagram illustrates two standard shares (for example, accounting
and IT) on the same file server.
Standard shares and exports can store files in the root of the directory.
Accessing the directory, using ls in this case, mounts the automatically created
top-level directory export. If you run df again you see two additional mount
points:
# df | grep project
1.1.1.10:/projects 1073741824 839455744 234286080 79% /projects
1.1.1.11:/projects/project1 1073741824 839455744 234286080 79% /projects/project1
1.1.1.12:/projects/project2 1073741824 839455744 234286080 79% /projects/project2
These additional mount points allow a different FSVM to serve each export.
This behavior introduces additional steps when you delete a top-level directory.
You can delete the project2 directory from any NFS client with the export
mounted if you sign on as a user with the appropriate permissions. There are
three steps to deleting a distributed share export.
First, delete the contents of the share:
rm -rf /projects/project2/*
At this point you’ve deleted the top-level directory and it becomes inaccessible
to other clients mounting the export. Processes that access the export after
you delete it receive a Stale File Handle error.
Distributed shares with NFSv3 clients behave differently than those with NFSv4
clients. NFSv3 connections don’t automatically mount the top-level directories
of distributed shares. Because the top-level directories aren’t submounted, you
can rename and delete them without the additional steps required for NFSv4.
Nested Shares
Nutanix Files 3.2 introduced support for nested shares. Using a nested share,
you can create a folder in an existing standard or distributed share and turn
that folder into a directly accessible share. Both SMB and NFS protocols
support nested shares. Nested shares inherit some attributes from the parent
share, but you can modify other attributes.
• Nested share inherited attributes:
› Protocol type
› Maximum size
› Self-service restore
› Quota policy
• Nested share modifiable attributes:
› SMB access-based enumeration (ABE)
› NFS authentication and access
› NFS advanced settings
Connected Shares
The connected shares function mounts shares and exports as subfolders in
other shares and exports. You can create a folder at any level of the folder
hierarchy and mount either a standard or distributed share into that folder.
Starting with Files 3.8, the share used as the parent hosting the folder can be a
standard, distributed, or nested share.
WORM Shares
Write once read many (WORM) shares offer retention policies to ensure
files aren’t inadvertently deleted. WORM shares are provided by failing file
operations, which can cause file modification or deletion.
Note: Not all applications are expected to work on a WORM share. Only applications that don’t
rely on operations that modify or delete files will work normally.
Share-level WORM support was added with the release of Files 4.1. WORM
shares automatically commit files into a WORM state, specified by configuring
a cool-off interval for the share. When a regular file in the share hasn’t been
changed—that is, the change time (or ctime) remains unchanged—for the
duration specified by the cool-off interval, it transitions into a locked state.
From there, the file goes into retention, also called commit. The file remains
in retention until the end of its retention period, if one is specified; otherwise,
the file is retained for the share’s default retention period. When the retention
period ends, the file then transitions into an expired state. The purpose of the
expired state is to facilitate the file’s deletion.
Note: You can only enable share-level retention (WORM) while creating a new share.
A single FSVM and volume group serve each standard share or export.
Once you reach the limit of 10 volume groups per FSVM, new shares and
exports use existing volume groups of the same type. For example, if you
deploy a distributed share (15 volume groups) and 15 standard shares (creating
15 additional volume groups) on a three-node physical cluster, each FSVM hosts
10 volume groups: 5 for the distributed share and 5 for the standard shares. In
this situation, because each FSVM is serving the maximum of 10 volume groups,
the next share created uses an existing volume group.
Every file server maps one-to-one to a container. Using this mapping, you
can manage compression and erasure coding individually for each file server
deployed. Inline compression and erasure coding are enabled by default to save
capacity. We don’t recommend turning on deduplication.
Note: Use distributed shares for most of your use cases to ensure that user connections and data
ownership are distributed across the FSVMs in the cluster.
From Files 3.6 onward, you can control compression on a share-by-share basis.
When you create a share, compression is enabled by default.
Directory Layout
Nutanix Files can store millions of files in a single share and billions of files
across a multinode cluster with multiple shares. To achieve good response
times for environments with high file and directory counts, give some thought
to directory design. Placing millions of files or directories in a single directory
slows file enumeration, which may impact some applications.
For performance-sensitive applications, Nutanix recommends that you limit
directory width—the number of files or directories in the root of a folder—
to 100,000 objects. Increasing FSVM memory to cache metadata can help
improve performance for environments with high object counts where directory
enumeration is common.
Nutanix also recommends that you limit the number of top-level directories in
distributed shares. The recommended number of top-level directories depends
on the memory assigned to the FSVMs in the cluster. Your directory count
shouldn’t exceed 3,000 times the FSVM memory of one node. For example,
for a three-node cluster with 12 GB of memory assigned to each FSVM, the
maximum top-level directory count is 3,000 × 12 = 36,000.
Load balancing through volume group redistribution may not always improve
performance. For example, if clients target a low-level share directory that
can’t be further distributed among FSVMs, performance doesn’t improve. In
such cases, Nutanix Files supports scaling up the FSVMs by adding vCPUs and
memory. Scaling up is seamless to end users.
Note: We recommend a scale-up operation for performance optimization if SMB connection
limits reach 95 percent utilization over a two-hour time window.
There is a brief outage during volume group migration and FSVM scale-out if
you aren’t using shares with continuous availability enabled. The file share or
export requires a client reconnect after migration and scaling out. Today most
clients try to reconnect for 50–60 seconds, which limits the overall impact.
Load balancing also occurs for each vDisk in a volume group. Nutanix Volumes
requires the administrator to configure an iSCSI data service IP address.
The data service IP address is a highly available virtual IP address used as
an iSCSI discovery portal. Each vDisk in a volume group represents its own
iSCSI target that any CVM in the cluster can own. Nutanix Volumes uses iSCSI
redirection to place and automatically load balance these sessions as needed.
The load balancing feature for Nutanix Volumes is called the Acropolis Dynamic
Scheduler (ADS).
For more detailed information, refer to the Nutanix Volumes best practices
guide.
High Availability
Nutanix designed Files to recover from a range of service disruptions, including
when a local CVM or FSVM restarts or fails.
Figure 18: Nutanix Volumes Load Balancing for File Server Volume Groups
When the failed CVM returns to operation, the iSCSI session fails back. In
the case of a failback, the system signs the FSVM off and redirects it to the
appropriate CVM.
Node Failure
When a physical node fails completely, Nutanix Files uses leadership elections
and the local Minerva CVM service to recover. The FSVM sends heartbeats to
its local Minerva CVM service once per second, indicating its state and that
it’s alive. The Minerva CVM service keeps track of this information and can act
during a failover.
When an FSVM goes down, the Minerva CVM service unlocks the files from
the downed FSVM and releases the external address from eth1. The downed
FSVM’s resources then appear on a running FSVM. The internal Zookeeper
instances store this information so that they can send it to other FSVMs if
necessary.
The Nutanix Files Zookeeper instance tracks the original FSVM’s ownership
using the storage IP address (eth0), which doesn’t float from node to node.
Because FSVM-1’s client IP address from eth1 is now on FSVM-2, client
connections persist. The volume group and its shares and exports are
reregistered and locked to FSVM-2 until FSVM-1 can recover and a grace period
has elapsed.
The failover process typically takes between 30 and 180 seconds, and
client operations can be impacted during this time. SMB shares enabled for
continuous availability and hard-mounted NFS shares experience I/O delays but
maintain connections during high-availability events.
If an FSVM is offline for 30 minutes or longer, the Nutanix Files cluster
redistributes core cluster services (such as Zookeeper) owned by the offline
node. The cluster needs at least three online FSVMs in order to move services.
Moving cluster services helps the Files cluster rebuild itself smaller and survive
subsequent node failures.
When FSVM-1 comes back online and finds its shares and exports locked, it
assumes that a high-availability event has occurred. After the grace period
expires, FSVM-1 regains control of the volume group through the Minerva CVM
service. Once the failed FSVM comes back online, the cluster automatically
switches back to the larger size.
To summarize, the process Nutanix Files goes through to reinstate control is as
follows:
1. Stop SMB and NFS services.
2. Disconnect the volume group.
3. Release the IP address and share and export locks.
4. Register the volume group with FSVM-1.
5. Present new shares and exports to FSVM-1 with eth1.
Single-Node Environments
Nutanix Files clusters that only have one FSVM rely on hypervisor-based high
availability to maintain services. If the physical node that owns the FSVM fails,
the VM restarts on another physical node. Client connections are disrupted until
the FSVM restarts.
You can continue deploying additional FSVMs if you have free nodes; you can
also deploy multiple file servers.
Nutanix Files uses DFS referrals to direct clients to the FSVM that owns the
targeted share or top-level directory. The following diagram shows what
happens behind the scenes when a client sends a file access request.
1. When the user Dodi accesses their files, they click on a shortcut that triggers
a DNS request. The DNS request is first sent for the file server name.
2. Using DNS round robin, DNS replies with an FSVM IP address. In this
example, the IP address for FSVM-1 returned first.
3. The client sends a create or open request to FSVM-1.
4. The \homes\dodi folder doesn’t exist on this file server, so a
STATUS_PATH_NOT_COVERED is returned.
5. The client then requests a DFS referral for the folder.
6. FSVM-1 looks up the correct mapping in the file server’s Zookeeper and
refers the client to FSVM-2.
7. A DNS request goes out to resolve FSVM-2.
8. The DNS request returns the IP address of FSVM-2.
9. The client gets access to the correct folder.
Managing Shares
You can manage Nutanix Files shares the same way you manage traditional
file servers. For standard shares and top-level directories in distributed shares,
you can assign permissions with native tools such as Windows Explorer. NTFS-
level permissions, also called Windows access control lists (ACLs) or NTACLs,
manage all file and folder access.
The distributed share has some special requirements because of how we use
DFS referrals. DFS referrals have a single namespace even though the data
contained in the share may be spread out over many FSVMs. Typically, to
delete a user’s home directory, you select the top-level directory and delete
the entire directory subtree. However, when using a distributed export or
share you can’t just delete the top-level directory because it’s a separate share
created internally by Nutanix Files. Because you can’t directly delete a top-level
directory, removing it requires additional steps.
There are two options for renaming or deleting distributed share folders:
1. Find out which FSVM is hosting the folder and rename or delete the folder
directly using the FSVM as the UNC path.
2. Use the Nutanix Files MMC snap-in to manage top-level directories. Any file
server administrator can perform the MMC operations; you don’t need to
assign privileges manually. Establish a connection to the Files namespace
and perform the following SMB share management tasks:
a. Create, delete, rename, and change permissions for top-level directories.
b. Change NTFS permissions for shares.
Note: Use the Nutanix Files MMC when you modify NTFS permissions at the root of a distributed
share. The Files MMC ensures that permissions are propagated to the top-level directories based
on inheritance.
You can also manage open file locks and user sessions with the Windows
Shared Folders MMC.
Access-Based Enumeration
Access-based enumeration (ABE) is a Windows feature (SMB protocol) that
filters the list of available files and folders on the file server to include only those
the requesting user can access. This filter helps both users and administrators,
saving time for the user and worry for the administrator concerned about users
accessing files not meant for them.
ABE doesn’t control security permissions, and running ABE has associated
overhead. Every time a user requests a browse operation, ABE must filter out
objects the user doesn’t have permission for. Even if the user has permission to
access all contents of the share, ABE still runs, causing additional CPU cycles
and increased latency.
Note: Home directories (using a distributed share) are a great example of where you shouldn’t
enable ABE. As most users get their home mapping when they sign on and always have access
to their own content, we don’t recommend enabling ABE for home directory use cases.
SMB Signing
SMB signing is a feature of the SMB protocol that enables a digital signature
against each network packet. Digital signing helps ensure the packet’s origin
and authenticity and prevent tampering, such as eavesdropping attacks against
data in-flight.
Nutanix Files supports SMB signing and honors the signing request as
configured from the SMB client. Nutanix Files doesn’t require a configuration
setting; just enable SMB signing from the client and Nutanix Files negotiates the
appropriate setting.
The SMB protocol uses AES-CMAC to compute signatures for SMB signing.
This computation process can impact performance against the SMB share
where you enabled signing. With Nutanix Files 3.2 and later versions, the
signature computation uses the Intel processor AES-NI instruction set. The Intel
processor hardware acceleration helps reduce the overhead associated with
SMB signing.
Note: SMB signing decreases the maximum performance possible for SMB shares.
SMB Encryption
Nutanix Files 3.6 introduced support for in-flight encryption of SMB data
between clients and shares. SMB encryption is disabled by default with Nutanix
Files. You can enable encryption as needed on a share-by-share basis.
Note: SMB encryption decreases the maximum performance possible for SMB shares.
DFS Namespace
DFS Namespaces (DFS-Ns) are commonly used to logically organize shared
folders on different servers or in different geographic locations into a single
namespace. These shared folders appear to the user as a unified hierarchical
directory they can navigate using any Windows client. The server names and
their locations are completely hidden from the user, enabling large scale-out
architectures.
The most common architecture is an Active Directory–integrated DFS-N, where
the namespace is hosted on two or more domain controllers (namespace
servers) and the file data is stored on member servers. Nutanix Files supports
the use of DFS-N when you use Nutanix Files as a member server and Nutanix
Files shares as folder targets in the namespace. When you use Files 3.5.1 or later
versions, you can use either distributed shares or standard shares.
DFS-N is also commonly used in active-active replication scenarios. DFS-
N allows multiple file servers hosting the same data to support a common
folder. DFS-N helps provide site affinity, based on Active Directory, for users
to connect to the file server acting as a folder target. You need a replication
engine to maintain the data when active-active scenarios are present. Nutanix
supports the use of Peer Software as the replication engine for active-active
scenarios. To learn more about the Peer Software and Nutanix integration, see
the PeerGFS and Nutanix Files datasheet.
When you deploy Nutanix Files for NFS, you can select Active Directory or
LDAP or leave it unmanaged. The following diagram shows what happens
behind the scenes when a client sends a file access request using NFS.
1. When the server wants to access files, the client first sends a DNS request
for the file server name.
2. Using DNS round robin, a DNS reply returns with an FSVM address. In this
example, the IP address for FSVM-1 returned first.
3. The client sends a create/open request to FSVM-1.
4. The \Logs mount doesn’t exist on this file server, so it returns
NFS4ERR_MOVED.
5. The client then requests a GETATTR(FS_LOCATIONS).
6. FSVM-1 looks up the correct mapping in the file server’s Zookeeper and
refers the client to FSVM-2.
7. A DNS request goes out to resolve FSVM-2.
8. The DNS request returns the IP address of FSVM-2.
9. The client gets access to the correct mount point.
NFSv3
Nutanix Files supports NFSv3. Each file server has NFSv3 enabled by default,
but you can manually disable it. All NFS exports have NFSv3 enabled or
disabled based on this file server setting. Files 3.5 supports LDAP and
unmanaged exports with NFSv3 but doesn’t support Active Directory and
Kerberos for NFSv3.
Note: You must mount exports with the TCP protocol if clients use NFSv3.
Nutanix Files with NFSv3 includes support for both distributed and standard
exports. Unlike NFSv4, NFSv3 doesn’t support DFS-like referrals. Clients aren’t
Multiprotocol
Beginning with Nutanix Files 3.5.1, you can create file shares that are
accessible from both SMB and NFS clients, referred to as multiprotocol shares.
Multiprotocol shares allow simultaneous read access to the underlying file data
from either protocol. Write access can also occur from either protocol, but not
simultaneously to the same file. Authentication support for multiprotocol shares
includes all protocols and authentication options available with Nutanix Files
(such as Active Directory for SMB and LDAP, AUTH_SYS, AUTH_NONE, and
Active Directory support for NFS).
You manage all access control for a multiprotocol share using the native
protocol. For SMB, the native protocol is Windows ACLs, and for NFS, it’s Unix
mode bits. When non-native protocol access occurs, Nutanix Files maps user
access to the permission applied with the native protocol. A mapping must
exist between the user accounts using the non-native protocol and the user
accounts with permission applied through the native protocol.
If you use Active Directory for both SMB and NFS with Kerberos, you don’t
need any explicit mappings because both have the same users and groups. You
can use Identity Management for Unix (RFC 2307) to map NFS users to your
Active Directory users with attributes. For LDAP or unmanaged accounts, or
for Active Directory without Kerberos, you must create a mapping between the
users and groups in Active Directory. You can use Prism or the nCLI to manage
Nutanix Files user mappings.
You can configure a default index to map all non-native users and groups to
a specific native user or group. You can also use a rule-based mapping for
Active Directory and LDAP users—specifically, a template where the SMB name
matches the NFS username. Additionally, you can configure explicit mapping,
which overrides rule-based mapping. Explicit mapping consists of two mapping
subcategories: one-to-one mapping lists and wildcard mapping. You can use
the one-to-one mapping list to manually enter or upload a CSV file that maps
users or groups across protocols. Use wildcards for many-to-one mapping. You
can also deny share access for a specific user or group. For more details on
how to create user mappings, see the user mapping section of the Nutanix Files
guide.
The following table shows the user mapping requirements depending on the
chosen directory service and authentication type.
Table: User Mapping Requirements
SMB Directory NFS Directory Export Supported User Mapping
Service Service Authentication Required
Type
AD AD Kerberos5* Yes No
Quotas
Administrators can configure the default, user, and group quota space for any
share. The default level is the quota limit for every user unless the administrator
specifies otherwise. A user-level quota policy sets a specific amount of storage
for a single user. For example, if an administrator allocates 1 GB, the user
can’t take more than 1 GB. A group-level quota policy extends a user policy
to include all users for an entire Active Directory group, where each user can
use the assigned quota value. For example, if the administrator sets a group’s
quota to 10 GB, then each member of that group can use 10 GB. You can
assign quotas for SMB, NFS, or multiprotocol-enabled shares. In the case of
multiprotocol shares, administrators apply quotas using the native protocol.
User or group policies are enforced for the non-native protocol based on the
user mapping.
Notifications
Administrators can configure Prism to send email alerts to the user and to other
recipients using the same engine that sends cluster alerts. Designated users
receive email notifications when the quota is near maximum consumption—a
warning email at 90 percent and an alert email at 100 percent. You can also add
departmental share owners to the email notification list so they know they may
need to take action. With the Files 4.0 release, you can customize the email
template used for notification.
The following table shows the order of precedence when dealing with quotas
for a share.
Table: Order of Precedence for Quotas
Order of Precedence Policy
1 User policy
2 Group policy (the group policy with the
highest quota wins)
3 Default user policy
Enforcement
The administrator can also configure enforcement types for each quota-level
category. Enforcement types determine if a user or group can continue to
use the share after they’ve consumed their quota. A hard enforcement type
prevents the user from writing on the share when they reach their quota limit.
A soft enforcement type allows a user to write even if they exceed the quota
limit. Under either enforcement type, users over their quota receive an email
notification every 24 hours until they resolve the issue.
A comma-separated list of file names and extensions defines the blocked file
types. File types blocked at the server level apply to all shares. When you define
a list of file types at the share level, the share-level setting overrides the server-
level setting.
When you attempt to create a file with or rename a file to a blocked pattern,
an access denied message appears. You can read, edit, or delete existing files
created prior to a blocked file type policy. You can also use File Analytics
to discover if any unwanted file types exist on a file server before you apply
the policy. With version 3.0 or later, File Analytics can also apply a list of file
patterns matching known ransomware variants.
Note: When you use Data Lens to manage your ransomware file name patterns, you can’t see
the patterns from within the file server or share specific blocking lists.
File Analytics
Nutanix Files offers a rich auditing API that applications can subscribe to and
receive real-time notifications of file-related events, including file creation,
deletion, read, write, and permissions changes. A common use for these APIs is
forwarding such events to a syslog server for retention and audit trails. While
logging audit trails is useful, we needed to simplify insights into this data, so
we developed Nutanix File Analytics to consume this native auditing API while
delivering additional insights to the underlying data and user activity.
You can deploy File Analytics using Prism: download the Analytics bundle and
perform a one-click deployment. Once deployed, launch the File Analytics
page and enable analytics against the desired file server instances running on
your cluster. Enabling analytics requires a user account with administrative
permissions to do an initial scan of the file server.
Intelligent Insights
File Analytics scans and then continuously captures all file activity for
registered file server instances. This logging helps form a repository of
information so administrators can review what operations have occurred
against specific data and by specific users. File Analytics analyzes logged
events to provide an initial dashboard of information:
• Capacity trend, which shows what’s being consumed and how it’s changed
over time.
• Data age, which is the calculation of the last time a file was modified and the
percentage of data at varying age ranges.
• Anomaly alerts, which show all file operations that exceed a given anomaly
threshold, like the deletion of many files.
• Permission denials, which are the number of permission-denial events for
specific users over the selected time range.
• File distribution by size, which shows the number of files in a given size
range.
• File distribution by type, which details storage consumption by file type, such
as log files, image files, and video files.
• The top five active users based on total operations over the selected time
period.
• The top five accessed files based on total operations over the selected time
period.
• File operations, which detail the most frequent types of operations, like file
create, read, or write, over a selected time period, including the trend over
that time.
The dashboard provides a quick and easy health check so you better
understand capacity trends, data age, and file activity that could indicate
malicious activity.
Audit Trails
Audit trails allow you to search for a specific file or a specific end user to find
all file or user activity for a given time. You can search for a given file or user
based on wildcards. The frequency and types of operations by users and
against files, including the time they occurred, are displayed over your specified
time period. You can further filter the audit trail based on operation type, such
as open, read, write, delete, and other events.
The ability to search user and file activity helps you monitor access activity
against sensitive data. You can also download the queried activity to a JSON-
or CSV-formatted file for further reporting activities.
Anomaly Detection
With File Analytics, you can define anomaly alerts that represent specific
operations as run by an individual or against the file server as a whole.
As anomaly events occur, the email addresses defined in the anomaly rule
receive detailed information. You can also monitor each anomaly alert and track
it over time with the usage anomalies dashboard. The dashboard shows you
the users who caused the anomalies, the folders affected by the anomalies, the
types of anomalies, and trends.
Strange access patterns that trigger anomalies can indicate malicious user,
virus, or ransomware activity. Seeing this activity and being alerted in real time
can help you stop these operations and better understand user access patterns
across your organization.
Ransomware Intelligence
Ransomware is a persistent concern that requires multiple security controls
and software layers to mitigate. Nutanix Files has long supported centralized
antivirus scanning through ICAP (Internet Content Adaptation Protocol)
with security vendors like Trend Micro, McAfee, BitDefender, and Symantec.
Nutanix offers a comprehensive approach to ransomware across our portfolio
of products that you can read about in more detail in our Ransomware Threat
solution brief. To seriously respond to ransomware, a solution should prevent
the infiltration of ransomware and malware, detect any infection attempts, alert
the organization and initiate defensive measures, and provide a comprehensive
strategy for recovery, if necessary.
Nutanix Files and File Analytics have many of the core features required to
help detect, protect, analyze, and recover from ransomware. File Analytics
3.0 begins the journey to combine these technologies into a comprehensive
interface, built so you can manage your ransomware strategy with Nutanix
Files. It starts with a dedicated ransomware dashboard in File Analytics that
summarizes any detected vulnerabilities, including the impacted shares and
clients that may be compromised. The dashboard also shows you whether
your shares are protected with SSR snapshots. You can enable SSR against the
unprotected shares from this interface.
Nutanix Files can block file creation or file rename operations for specific file
extensions. Files 3.8 extended this feature to support wildcards with file names
and file extensions. When you enable ransomware protection in File Analytics,
Files automatically adds the names and extensions of known ransomware
variants to the blocking list. If anyone attempts any file creation or file rename
event involving these blocked file types, the event appears as a vulnerability
that Files reports on the dashboard and emails to the designated users.
Custom Reporting
File Analytics captures real-time user audit data and file metadata for Nutanix
Files environments. The 3.0 release of File Analytics enabled you to mine this
data more effectively by introducing a custom reporting page, so you no longer
needed to rely on the information available on the dashboard or through audit
search.
When creating a new report, you first select an entity, such as files, folders,
audit events, or users. You can then choose which attributes to filter with
customizable values. For example, you can choose attributes like file size
greater than or less than a given value, file age based on access time or
creation date, or all audit events within a given time range. Finally, you can
choose which columns of data to include in the report, like user and file names,
paths, clients, operation types, and other details associated with the entity.
Once you define the report, you can save it and run it again at any time.
Smart Tier
Administrators with large scale Network Attached Storage (NAS) environments
frequently look for ways to improve storage efficiency, simplify administration,
and decrease cost. With Nutanix Files 4.0, we added a native tiering framework
called Smart Tier to address these requirements. Smart Tier enables you
to move cold or rarely used data to lower-cost storage while maintaining a
single namespace. You can tier data to any qualified S3 API compliant target,
including on-premises solutions or the public cloud. Smart Tier helps hybrid
multicloud environments handle unstructured data to retain long term, save
money, or use a virtually limitless pool of local storage for your Nutanix Files
workloads.
Smart Tier maintains a file stub in the SMB or NFS share path where clients can
perform inline reads to access the data. You can also recall files automatically,
based on access patterns, or with manual recall operations. Through Nutanix
Data Lens, you can configure tiering policies and set age- and capacity-based
thresholds, manual or automatic tiering schedules, and recall settings.
Data Lens tracks the access patterns for all shares and exports to determine
the age of each file for a given file server instance. You can define hot, warm,
and cold data age categories to match your requirements. Using the data age
explorer, you can view the file age on a share-by-share basis to understand
what data is included in a given tiering policy.
Tiered Data
When data reaches the configured age and the file server is at its capacity
threshold, Smart Tier moves file data to the target. Smart Tier marks the
previously consumed space as free, creating additional space in the share for
new or recalled data.
Tiered files appear as regular files in their folder paths. SMB shares have an
offline attribute set, which in Windows Explorer shows an X beside the file icon.
You can see the file size (size on disk), which represents just the metadata, and
the actual file size, which represents the tiered data. You can read (which here
means retrieve the data inline from) tiered files at any time, but you can’t write
to a tiered file directly. You need to either retrieve the file or make a separate
copy to edit it.
Through Data Lens you can see the hot, warm, cold, and archived data
associated with tiering. You can also define cost thresholds that help you see
the current and potential cost savings based on your tiering policies.
Hypervisor-Specific Support
Nutanix supports ESXi and AHV for Files. For ESXi support, you need vCenter
credentials to deploy Nutanix Files and to create DRS rules to make sure the
FSVMs are on different nodes. You must register vCenter with the Nutanix
cluster instances where you deploy Files. The deployment process generates
the DRS rules for AHV automatically.
Data-at-Rest Encryption
Nutanix Files supports data-at-rest encryption (DARE) with self-encrypting
drives (SEDs) or software-defined DARE using AOS. Official support for AOS
software encryption with Nutanix Files begins with AOS 5.10.2 in combination
with Nutanix Files 3.5.
For AHV, you enable software encryption at the cluster level. For ESXi, you
enable software encryption at either the cluster level or the storage container
level. To enable storage container–level software encryption with Nutanix Files,
you must use nCLI:
<ncli> storage-container edit enable-software-encryption=true
name=<Files_Container_Name> force=true
Self-Service Restore
Administrators can enable SSR at any time for SMB or NFS shares. Windows
Previous Version in each folder exposes SSR for SMB shares. SSR for NFS
shares is exposed as a hidden snapshot directory in each folder.
SSR allows you to view snapshots of shares while the share is in use. The share
snapshots are read-only, point-in-time copies.
You can view and restore removed or overwritten files, which means you can
choose a share snapshot from the same file at different times during the file’s
Note: You can configure a maximum of 50 total snapshots in the SSR schedule.
Once you’ve protected Nutanix Files, all future operations on it (such as adding
or removing FSVMs or adding or deleting volume groups) automatically update
the corresponding consistency group in the protection domain.
Cloning
Because Nutanix Files cloning doesn’t affect the original Files cluster, it offers
improved support for a variety of use cases:
• Backups at the primary and secondary sites.
• Disaster recovery at the secondary site.
• File server recovery from a specific point in time.
• File server creation at the primary or remote site for testing or development.
• File server clone copies.
Files uses Nutanix native snapshots to clone entire file servers. The clone is a
thin copy that consumes minimal storage space. File server clones reside on the
same container as the original and maintain the original security permissions.
During the clone process you can specify new IP addresses and give the cloned
file server a new name.
Files Manager 2.0 provides a data protection menu where you can configure,
monitor, and orchestrate failover and failback operations for Smart DR.
Further, for the file systems that support the shares performing replication, you
can now set policies on a share-by-share basis to manage your recovery point
objective (RPO) at the share level instead of at the file server level.
Replication Policies
A replication policy defines the share or group of shares you want to replicate.
The policy also defines the source and target file servers and the replication
frequency. You can create multiple policies as needed and specify a default
policy for any newly created shares.
Monitoring
Files Manager in Prism Central provides SLA monitoring and job replication
status at several levels. You can use the RPO compliance overview provided on
the summary page to quickly understand if replication is keeping up with the
defined policy.
You can also view each replication job to monitor its completion percentage,
start and end times, amount of data synchronized, and average network
bandwidth usage.
With a planned failover, you can choose to begin replicating in the opposite
direction automatically. Replicating after failover helps maintain service level
agreements (SLAs) and RPOs during your failover testing or disaster avoidance
operations.
SSR Interoperability
With Files 4.0, Smart DR supports replicating the snapshots between the
source share and its target. The target share retention schedule doesn’t have
to match the source retention schedule. For example, if you’ve configured
the source file server to maintain the last two hourly snapshots, then you
can configure the target file server to maintain the last ten hourly snapshots,
providing a longer retention window. Alternatively, you can match retention
schedules to ensure that the same SSR copies exist in both sites for any failover
event.
SSR schedules must have the same frequency type between the source and
target. For example, if you have daily snapshots scheduled on your source,
you need to schedule daily snapshots on your target if you want them to be
replicated and retained.
5. Third-Party Integration
Antivirus
To protect users from malware and viruses, you need to address both the client
and the file server. Nutanix currently supports third-party vendors that use
Internet Content Adaptation Protocol (ICAP) servers. ICAP, which is supported
by a wide range of security vendors and products, is a standard protocol that
allows you to integrate file and web servers with security products. Nutanix
chose this method to give customers wide latitude in selecting an antivirus
solution that works best for their specific environments.
The following workflow is for an ICAP-supported antivirus solution:
1. An SMB client submits a request to open or close a file.
2. The file server determines if the file needs to be scanned, based on the
metadata and virus scan policies. If a scan is needed, the file server sends the
file to the ICAP server and issues a scan request.
3. The ICAP server scans the file and reports the scan results back to the file
server.
4. The file server takes an action based on the scan results:
a. If the file is infected, the file server quarantines it and returns an access
denied message to the SMB client.
b. If the file isn’t infected, it returns the file handle to the SMB client.
The ICAP service runs on each Nutanix Files file server and can interact with
more than one ICAP server in parallel to support horizontal scale-out of the
antivirus server. The scale-out nature of Files and one-click optimization greatly
Nutanix Files sets scanning defaults across the entire file server. You can enable
scan on write and scan on read. Scan on write begins when the file is closed,
and scan on read occurs when the file is opened. You can also exclude certain
file types and sizes. Share scan policies can override any defaults set for the file
server.
For each ICAP server, Nutanix Files spins up no more than 10 parallel
connections per FSVM and randomly distributes the file scanning among all the
ICAP servers. With heavier workloads, which may involve many scan requests
and use all connections, you can give the scan servers more processing power
to scan more files. As soon as the current scan finishes, the scan server picks up
the next file from the queue, which keeps the number of active connections at
10.
Once Nutanix Files quarantines a file, the administrator can rescan the file,
remove it from quarantine, or delete it. You can search quarantined files if you
need to restore a file quickly.
If your antivirus vendor doesn’t support ICAP, you can scan the shares by
installing an antivirus agent on a Windows machine, then mounting all the
shares from the file server. With this approach you can schedule scans for
periods of low usage. At the desktop or client level, you can set your antivirus
solution to scan on write or scan only when files are modified. You can
configure high-security environments to scan inline for both reads and writes.
See the Compatibility and Interoperability Matrix on the Nutanix portal for the
latest list of qualified ICAP server vendors with Nutanix Files.
future auditing needs. Several vendors have integrated with this operations
monitoring API:
Peer Software
Peer Software has integrated with Nutanix, including the file operations
monitoring API, to support active-active deployments with real-time
replication and file locking between heterogenous environments. To learn
more about the Peer Software and Nutanix integration, see the PeerGFS
and Nutanix Files datasheet.
Netwrix
Netwrix has integrated with the Nutanix file operations monitoring API.
Netwrix provides comprehensive visibility into changes and data access
across Nutanix Files instances. Netwrix offers an add-on for Nutanix Files.
See netwrix.com for more details.
Varonis
Varonis version 8.6 introduces support for Nutanix Files version 3.7.1 and
higher. This support includes Varonis’ entire platform, DatAdvantage,
DatAlert/DatAlert Analytics, DataPrivilege, Data Classification Engine,
DatAnswers, Automation Engine, and Data Transport Engine.
Intelligent Backup
Traditional NAS backup approaches like Network Data Management Protocol
(NDMP) have core limitations. Problems like complexity, performance (including
the need for periodic full backups), and scale led Nutanix to develop a more
modern backup solution: a changed file tracking (CFT) API for third-party
backup vendors. This API shortens backup times because it doesn’t perform
a metadata scan across your file server, which could contain millions of files
and directories. The CFT API promotes scalability, as backups can occur across
multiple FSVMs in parallel. It also ensures that customers aren’t locked into any
one solution, as backups taken using the Nutanix Files API can be restored to
other vendors.
CFT uses internal snapshots to track differences between backup windows. The
backup software vendor requests the creation of these snapshots, and Files
returns a list of the changed files in the form of a share. The backup software
proceeds to mount the share onto the backup server to be read for backup.
Backup software can specify multiple shares and their respective snapshot
information. Nutanix Files returns the list of URLs that map to the number of
client streams that can start in parallel. Because shares are distributed evenly
across the FSVMs based on load, the backup software can take advantage of all
the FSVMs to drive throughput.
Multiple software vendors have integrated with the Nutanix Files CFT API,
including:
• HYCU Backup and Recovery
• Commvault, version 11, service pack 15 (as of CFT API release 3.5)
• Veritas Netbackup 9.1
There are multiple ways to back up Files shares with software that doesn’t
support CFT. One option is to run the backup application on a Windows
machine and map the UNC path of the share as a drive that needs to be backed
up.
There are also vendors that provide support for backing up file shares without
mounting to a guest VM. These applications can read directly from the UNC
path.
Both Veeam and Rubrik are validated as non-CFT backup solutions for Nutanix
Files. Refer to the Compatibility and Interoperability Matrix for the full list of
validated backup applications.
Note: Don’t try to back up the FSVMs. FSVMs are stateless, and data is stored in Nutanix volume
groups. Back up Nutanix Files from the share level. You can protect FSVMs using Nutanix
protection domains.
Because the system spreads different standard shares across the cluster,
try to back up multiple shares at the same time with multiple subclients.
The distributed share allows you to configure multiple data readers to drive
throughput.
was virtualized and configured with only eight vCPU. Adding more vCPUs to
the media agent achieved a shorter backup time.
6. Conclusion
Nutanix Files delivers on-demand performance and automated provisioning
to provide highly scalable file management. It reduces the administration and
configuration time needed to deploy and maintain your environment, providing
a public cloud experience within your private cloud.
AOS distributed storage and Prism make hosted file services highly resilient and
easy to use—for example, you can configure the native data efficiency features
of AOS, such as erasure coding (EC-X), for each individual file server. Prism
lets you administer network and resource management, Active Directory, fault
tolerance, and Nutanix Files share and export management all in one place,
vastly improving operational efficiency.
File Analytics gives you deep insight into your data, including storage
consumption trends, file sizes, file counts, and data age. Analytics also provides
end-to-end audit trails and anomaly detection. Ransomware intelligence helps
you detect, protect against, analyze, and recover from ransomware threats.
Because you can deploy Files on a new or existing Nutanix environment, as well
as in environments that are dedicated or mixed (with other services), you have
the flexibility to choose your architecture to take advantage of Files services.
Nutanix Files streamlines design, implementation, and maintenance, providing
an unrivaled experience for key use cases like end-user computing, video
capture, medical imaging archives, application data, and the many other
projects that store data in shared environments.
About Nutanix
Nutanix is a global leader in cloud software and a pioneer in hyperconverged
infrastructure solutions, making clouds invisible and freeing customers to
focus on their business outcomes. Organizations around the world use Nutanix
software to leverage a single platform to manage any app at any location
for their hybrid multicloud environments. Learn more at www.nutanix.com or
follow us on Twitter @nutanix.
List of Figures
Figure 1: Nutanix Files Instances Run as VMs.......................................................................................................... 8
Figure 18: Nutanix Volumes Load Balancing for File Server Volume Groups........................................ 27