Operational Shutdown

Before making any changes to the Solarwinds environment, such as patching, application or OS, a shutdown of the Solarwinds services is required.

Orion Service Manager can be located on the Solarwinds application server and is a better alternative to having to stop and start services manually.

To shutdown Solarwinds correctly click stop everything.

To startup Solarwinds correctly click start everything.

Operational Configuration – Daily

  1. Check for recurring alerts and events, determine what is causing the recurring alert/event, raise and resolve the issue with relevant service owner and alter thresholds accordingly.
  2. Check environmental appstack and application health overview for new critical or warning items. Raise and resolve.
  3. Check through nightly imported nodes, configure Appinsight and apply templates if required, speak to service owner to find out about new nodes and requirements.
  4. Remove nodes or unmanage nodes, no longer required, confirmed by service owner.
  5. Check NOC displays for updates due to new nodes being added or nodes being removed.
  6. Raise any particular alerting issues at the morning meeting.

Preventative Maintenance – Weekly

  1. Review and apply as needed Application Patches, Hotfixes to Solarwinds Application Server.
  2. Review and apply as needed Updates to OS, Hardware Drivers, etc.
  3. Review CPU/Memory/Disk on Monitoring application server and Monitoring SQL server and any other Solarwinds application servers.
  4. Check Active Diagnostics Bi-Weekly.
  5. Run Diagnostics for support purposes – Store elsewhere than application or database server (Other network location or offline storage).
  6. Take Snapshots (If virtualized).
  7. Review Enable/Disable Automatic Baselining settings as needed.
  8. Review Enable/Disable Automatic Dependencies as needed.
  9. Alerts should be running every 1 minute – unless the environment requires more regular alerting.
  10. Check in-house created Alerts’ raw SQL Queries against Estimated Execution Plan Time in SQL Server Management Studio to ensure queries don’t need refactoring.
  11. Review “Down” or “Unknown” Nodes or Applications for polling errors or that need to be unmanaged or removed from monitoring.
  12. Review Custom Property Utilization Across the Board for all Nodes, Applications, Interfaces, etc. – Fill in gaps where needed.

Database Maintenance Review – Monthly

  1. Verify Database Maintenance is Completing MS SQL Server running Orion Solarwinds database.
  2. Review C:\ProgramData\Solarwinds\Logs\Orion\swdebugMaintenance.log
  3. Ensure “Database Maintenance Complete” Message appears in a reasonable amount of time from when it begins at 2:15 AM every morning (by default).
  4. Ensure you have Database backups and you can restore from those backups.
  5. Index Rebuilds / Reindexing – Ensure this is happening, quarterly at a minimum.
  6. Review Total Database Size and/or large tables for uncharacteristic growth.
  7. Verifying Trap & Syslog Messages are being cleaned out (space reservations in SQL DB)

Settings and Configurations – Monthly

  1. Review Polling Engine Polling Status/Rates.
  2. Check Polling Completion (Polling Engine Status Report).
  3. Check Polling Settings (Consider Alert Time Frames vs. Polling Intervals). Reminder – Retention Syslog, Trap, Discovery, Downtime – Will increase DB Size.
  4. Review Logging Levels – Run Logadjuster.exe to see all logging settings.
  5. Ensure that Logging Levels are at the Defaults or specified standard settings.
  6. Review Logs with Quick Rotations/Timestamp Successions.
  7. Review Diagnostic Logs for Errors – Check against Success Center for common issues; escalate to Support if KB is unable to solve.
  8. Ensure Permissions are set appropriately (Run Orion Permission Checker).

For some great Solarwinds training videos click below.