#CiscoLive

HyperFlex Best Practices

Aaron Kapacinskas HX TME

BRKINI-2013

#CiscoLive
Agenda
• Release selection
• Installation BPs
• Operational BPs
• Upgrade
• Edge
• Stretch Cluster
• Cisco Solution Support

#CiscoLive BRKINI-2013 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 3
Release
Selection
 Planned Long Lived and Feature Releases
Q1CY19 Q2CY19 Q3CY19 Q4CY19 Q1CY20 Q2CY20 Q3CY20 Q4CY20 Q1CY21 Q2CY21 Q3CY21 Q4CY21 Q1CY22. Q2CY22

3.5(2x)
Regular Patches for Updates and Bug Fixes Support
Long Lived Security and Vulnerability Fixes Only
Only
Release
3.5(2a) 3.5(2b) ……………………………. 3.5(2g)

4.0(1x) Regular Patches for Updates

Support
Feature and Bug Fixes
Only
Release
4.0(1a) ……………… 4.0(1c)

4.0(2x) Support
Regular Patches for Updates and Bug Fixes Security and Vulnerability Fixes Only
Long Lived Only
Release
4.0(2a) 4.0(2b) ……………………………. 4.0(2g)

4.5(1x) Regular Patches for Updates

Support
Feature and Bug Fixes Only
Release
4.5(1a) ……………… 4.5(1c)

Note: The future release dates shown above are tentative and subject to
change
HX recommended SW releases bulletin - Link
HX release policy- Link

#CiscoLive BRKINI-2013 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 5
Selecting SW Release
Refer to the recommended release document
• Varies by configuration (e.g., SED AF) and features (e.g., Stretched
cluster)
• Recommendation for combination of HXDP, UCSM and ESX versions
• Doc updated based on CFDs, IFDs and release health in the field
Regarding Upgrades
• Change of recommended release doesn’t mean upgrade immediately
• For specific defects use release based on CDET info or release notes
• Urgent notifications of issues are communicated via field notices
Enable HX ASUP (remember to register an email that is monitored, in
the future we plan to send notifications there)

#CiscoLive BRKINI-2013 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 6
Best Practices – Release Notifications
Sign up for HX release notes
updates, new SW availability
and recommended release
updates using the Cisco
Notifications portal

#CiscoLive BRKINI-2013 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 7
Installation Best
Practices
Installation Best Practices
Pre-Install Considerations
• Be aware of limitations of the installation “location”
• For a given type of deployment (installer vs. Intersight) you may have limited
options for upgrade or expansion
• Use the preinstall checklist or the preinstall tool
• HyperFlex Edge Deployment Guide links to the HX Hardening Guide for
specific information regarding network port requirements
• Pay attention to the HX release notes and the HX recommended release
document

#CiscoLive BRKINI-2013 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 9
Infrastructure and Sizing Considerations

• Min upstream switch BW

• Always use RF3 in production
• Redundant ToRs • Enough capacity headroom for
• Port trunking Infra Availability better resiliency & performance
• NTP (local) and DNS
• Data protection plan (Backup,
outside the cluster disaster recovery, etc)
• vCenter outside cluster
Sizing
Factors
• Use HX sizer & Hx Profiler
• Smaller clusters - better fault
• N+1 or N+2 sizing for
tolerance, easier to upgrade,
sufficient perf headroom Allow for future expansion
• Size to make sure peaks can
be handled Perf Scale • Enable LAZ for larger clusters
• See Release notes for details of
• Pay attention to latency (in supported scale (config specific)
addition to IOPS/thruput)

#CiscoLive BRKINI-2013 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 10
Install Considerations - Continued
Networking Compute Misc

• Don't use VLAN 1 • Uniform compute-only • Multiple clusters under

• Keep data VLAN L2 nodes that closely match same FI - how to
isolated for security HX in capability – configure and avoid
• Always use unique symmetry conflict
storage VLAN per cluster • Boost Mode • Mixed CPU = EVC mode
• Deploy with IPs instead • Add 4 additional • Select “Clear Partitions”
of FQDN to reduce DNS vCPUs any time to to clean the disks
dependency all nodes • VDI or VSI – select one
• Do not skip jumbo • vCenter admin privs for hybrid systems
warning in installer - fix • No indication in • Run post install script
• Using 1500 – no warning Connect

#CiscoLive BRKINI-2013 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 11
Expansion Considerations
Cluster node or drive expansion:
• Pay attention to the BOM compatibility when expanding the cluster
• When doing both node and drive expansion – first add all the nodes and
then the drives
• Nodes add additional performance to assist with drive rebalance
• Pay careful attention to max scale limits when expanding
• See release notes

#CiscoLive BRKINI-2013 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 12
Configuration
and Operational
Best Practices
Configuration and Operational Best Practices
Resiliency
• Utilize burn-in or tests before production
• Test failovers
• Identify any process weaknesses
Datastores
• Don't put logs on HX datastore
• Only need one for most (regular cluster) use cases
• Same performance as multiple
• Less to manage
• Better dedup opportunity (clones)
• Fastest boot up time
• You may want 2 for HA heart-beating
• Adding addition datastores increases overall available queue depth
• Thin provisioning preferred - HX always thin-provisions capacity anyway

#CiscoLive BRKINI-2013 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 14
Configuration and Operational Best Practices
Security
• Follow the Hardening Guide
• Run the automated STIG script for best security posture
• Enable Secure Boot (HX 4.5.1a or higher)
• Apply ESXi security patches at any time if they are in the same release family
FI and vCenter
• Do not use HX host profiles or change any settings in UCSM or ESXi
• Set and forget
• Do not change NIC ordering in vCenter
• Multiple clusters on the same FI should have unique VLANS and MAC Pool
addresses
iSCSI
• MTU 9000 end to end (Don’t forget MTU 9000 on the Initiator)
• Enable BoostMode if possible
• Configure Initiator with HX iSCSI CIP when there is no multipath

#CiscoLive BRKINI-2013 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 15
Configuration and Operational Best Practices
General Considerations
• Future growth considerations need to be considered (trends)
• Intersight displays trends and capacity planning recommendations
• Reclaim space in VMs with high turnover – See Capacity Management
• Run Health Check - triaging issues or identify potential issues
• Built into HX 4.5.1a and above
• Intersight now has integrated HyperCheck capability for system health
• Be cautious of configuration drift
• Example: Changes to networking configuration
• Intersight HCL check
• Keep the cluster symmetric in expansion (compute nodes to match
converged)
• Stagger snapshots for scheduled snaps

#CiscoLive BRKINI-2013 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 16
Upgrade Best
Practices
Upgrade Best Practices
• Consult release selection guidelines and review release notes for desired target
• Make sure backups are current
• Use DRS in vCenter or you will have to manually vacate nodes.
• Understand the limitations on upgrading the environment non-disruptively
• Application, HX, Hypervisor, UCSM, component firmware
• App required to support vMotion/DRS or app level HA
• HX takes care of everything except the app. However, remember it is a rolling
upgrade and plan for enough perf capacity and headroom to accomplish that
• Understand timing requirements. Rolling upgrades on large clusters will take time
• Run Health Check - triaging issues before upgrades
• Do not use VUM and always use HX customized ESXi upgrade bundles

#CiscoLive BRKINI-2013 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 18
Edge Best
Practices
 Edge Best Practices
• Follow the guidelines in CVDs for general deployments
• e.g., Use dedicated 1GB CIMC port for Edge and go with 10GE
networking
• Understand expansion path (if any)
• 10 GBE for higher perf and future node expansion capability
• 1 GBE for clusters that will never be expanded and where TOR has
no 10 GBE
• Do not overburden Edge deployments. They are targeted for a specific
ROBO profile

#CiscoLive BRKINI-2013 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 20
 Edge Best Practices

• Reference the pre-install checklist (differentiated for 2 node) or the

Preinstall Tool
• Understand the advantages of Intersight
• API, Policy and Profiling tools for rapid cluster config
• Invisible cloud witness for 2 node deployments is a robust and
strong differentiator
• Cisco Intersight offers full life-cycle management
• 2 node must be deployed with Intersight
• 3 and 4 node can be deployed with Intersight or On-Prem Installer

#CiscoLive BRKINI-2013 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 21
Edge Best Practices – N:1 Replication
VMs requiring protection should be:
• Migrated into the backup datastore
• Created in the backup datastore
VMs are always restored into the backup datastore
• Consider deleting VMs that have been “test-recovered”
• Delete the snapshots of previously deleted “test-recovered” VMs
VMs that no longer require protection should be:
• Migrated out of the backup datastore
• Manually delete snapshots of VMs migrated out of the backup datastore
• stcli dp vm list –brief
• stcli dp vm delete --vmid

#CiscoLive BRKINI-2013 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 22
Stretch Clusters
Best Practices
Network - Overlay networks are in qualification
• OTV – supported
• VXLAN – testing
• NSX – not supported
• Keep the witness as fast as possible (<<100ms)
• Edge Invisible Witness is different from SC Witness
Datastores
• Create one on each site (affinity)
Failure Modes
• Survivability while maintaining online status requires a majority zookeeper
quorum and more than 50% of nodes (the witness counts as zk node)
• It is possible that the surviving site could tolerate a node or disk loss (in a
cluster greater than 2+2) if that node is not a zookeeper node, but it is not
guaranteed
• Be sure to fully recover after a site failure (manual migration back)

#CiscoLive BRKINI-2013 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 24
Best Practices
Installation and Operation – Read the SC WP
• FI models per site should be the same
• Use the Sizer whenever possible (takes RF4 into account)
• Should be able to run on one site - an enterprise solution
• Pre-plan port availability for firewalls
• Encryption - Third Party Software only
• Put the witness at a 3rd site (low latency)
• Be careful about witness upgrade – plan accordingly
• Expand symmetrically – update sites, expand
• 2:1 compute ratios when adding compute resources
• DRS and HA should be turned on in VC
• Adjust HA settings as needed in VC (follow the WP)
• HXDP and ESXi (4.0.2a+) rolling update supported, UCS FW is manual
• NVMe on HX220 in 4.0.2a and above

#CiscoLive BRKINI-2013 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 25
Additional Information
• Stretch Cluster White Paper
• HX with ACI CVD
• Admin Guide

#CiscoLive BRKINI-2013 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 26
Cisco Solution
Support
 Cisco Solution Support
The right kind of support service for Cisco Hyperflex and Data Center solution
environments

Overview Features Budget benefits

• Primary point of contact
• Priority response response by a Cisco solution Available on our standardized price list and
expert ordering tools for predictable costs on new
• 30 Minute or better response for Severity 1 and deployments and expansions of deployed
2 support requests solutions.
• Cisco and Solution Support Partner product
support team coordination
• Accountability for issue resolution
• Case management from first call to resolution Use cases
• Looks beyond original case to proactively
identify and address potential issues
Customers who:
Centralized support from Cisco® solution • Includes Cisco Smart Net Total Care® and
• Need a faster response time for high priority
experts who own issue management Software Support Services
cases
and resolution across the Cisco and • Want to call us for any issue with their Cisco
Solution Support Alliance Partner
components across a solution
Customer value solution, even if they haven’t isolated it to a
deployment. Cisco product
Innovate with confidence and achieve faster ROI: • Have HyperFlex servers part of larger data
• 44% faster resolution than product support 1 center with Cisco ACI, Security, third-party
Available for Cisco’s vast hardware and
software catalog as well as our data • 21% fewer issues 2 storage
center, collaboration, enterprise • 213% service ROI 2 • Replace Embedded ESXi
networking, Internet of Things, security, • 17% lower hardware OPEX 2 • Use HyperFlex for Virtual Desktop
and service provider solutions. Infrastructure or Data Protection Software
1 January 2019 Cisco internal study of 10,000 support cases; figure represents an average.
2 2017 IDC Business Value Study; read the executive summary.

#CiscoLive BRKINI-2013 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 28
Thank you

#CiscoLive
#CiscoLive