219 lines
9.0 KiB
Markdown
219 lines
9.0 KiB
Markdown
|
### NVMe-MI over SMBus
|
||
|
|
|
||
|
|
Author: Tony Lee <tony.lee@quantatw.com>
|
||
|
|
|
||
|
|
Created: 3-8-2019
|
||
|
|
|
||
|
|
#### Problem Description
|
||
|
|
|
||
|
|
Currently, OpenBMC does not support NVMe drive information. NVMe-MI
|
||
|
|
specification defines a command that can read the NVMe drive information via
|
||
|
|
SMBus directly. The NVMe drive can provide its information or status, like
|
||
|
|
vendor ID, temperature, etc. The aim of this proposal is to allow users to
|
||
|
|
monitor NVMe drives so appropriate action can be taken.
|
||
|
|
|
||
|
|
#### Background and References
|
||
|
|
|
||
|
|
NVMe-MI specification defines a command called
|
||
|
|
`NVM Express Basic Management Command` that can read the NVMe drives information
|
||
|
|
via SMBus directly. [1]. This command uses SMBus Block Read protocol specified
|
||
|
|
by the SMBus specification. [2].
|
||
|
|
|
||
|
|
For our purpose is retrieve NVMe drives information, therefore, using NVM
|
||
|
|
Express Basic Management Command where describe in NVMe-MI specification to
|
||
|
|
communicate with NVMe drives. According to different platforms, temperature
|
||
|
|
sensor, present status, LED and power sequence will be customized.
|
||
|
|
|
||
|
|
[1] NVM Express Management Interface Revision 1.0a April 8, 2017 in Appendix A.
|
||
|
|
(https://nvmexpress.org/wp-content/uploads/NVM_Express_Management_Interface_1_0a_2017.04.08_-_gold.pdf)
|
||
|
|
[2] System Management Bus (SMBus) Specification Version 3.0 20 Dec 2014
|
||
|
|
(http://smbus.org/specs/SMBus_3_0_20141220.pdf)
|
||
|
|
|
||
|
|
#### Requirements
|
||
|
|
|
||
|
|
The implementation should:
|
||
|
|
|
||
|
|
- Provide a daemon to monitor NVMe drives. Parameters to be monitored are Status
|
||
|
|
Flags, SMART Warnings, Temperature, Percentage Drive Life Used, Vendor ID, and
|
||
|
|
Serial Number.
|
||
|
|
- Provide a D-bus interface to allow other services to access data.
|
||
|
|
- Capability of communication over hardware channel I2C to NVMe drives.
|
||
|
|
- Ability to turn the fault LED on/off for each drive by SmartWarnings if the
|
||
|
|
object path of fault LED is defined in the configuration file.
|
||
|
|
|
||
|
|
#### Proposed Design
|
||
|
|
|
||
|
|
Create a D-bus service "xyz.openbmc_project.nvme.manager" with object paths for
|
||
|
|
each NVMe sensor: "/xyz/openbmc_project/sensors/temperature/nvme0",
|
||
|
|
"/xyz/openbmc_project/sensors/temperature/nvme1", etc. There is a JSON
|
||
|
|
configuration file for drive index, bus ID, and the fault LED object path for
|
||
|
|
each drive. For example,
|
||
|
|
|
||
|
|
```json
|
||
|
|
{
|
||
|
|
"NvmeDriveIndex": 0,
|
||
|
|
"NVMeDriveBusID": 16,
|
||
|
|
"NVMeDriveFaultLEDGroupPath": "/xyz/openbmc_project/led/groups/led_u2_0_fault",
|
||
|
|
"NVMeDrivePresentPin": 148,
|
||
|
|
"NVMeDrivePwrGoodPin": 161
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"NvmeDriveIndex": 1,
|
||
|
|
"NVMeDriveBusID": 17,
|
||
|
|
"NVMeDriveFaultLEDGroupPath": "/xyz/openbmc_project/led/groups/led_u2_0_fault",
|
||
|
|
"NVMeDrivePresentPin": 149,
|
||
|
|
"NVMeDrivePwrGoodPin": 162
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
Structure like:
|
||
|
|
|
||
|
|
Under the D-bus named "xyz.openbmc_project.nvme.manager":
|
||
|
|
|
||
|
|
```
|
||
|
|
/xyz/openbmc_project
|
||
|
|
└─/xyz/openbmc_project/sensors
|
||
|
|
└─/xyz/openbmc_project/sensors/temperature/nvme0
|
||
|
|
```
|
||
|
|
|
||
|
|
/xyz/openbmc_project/sensors/temperature/nvme0 Which implements:
|
||
|
|
|
||
|
|
- xyz.openbmc_project.Sensor.Value
|
||
|
|
- xyz.openbmc_project.Sensor.Threshold.Warning
|
||
|
|
- xyz.openbmc_project.Sensor.Threshold.Critical
|
||
|
|
|
||
|
|
Under the D-bus named "xyz.openbmc_project.Inventory.Manager":
|
||
|
|
|
||
|
|
```
|
||
|
|
/xyz/openbmc_project
|
||
|
|
└─/xyz/openbmc_project/inventory
|
||
|
|
└─/xyz/openbmc_project/inventory/system
|
||
|
|
└─/xyz/openbmc_project/inventory/system/chassis
|
||
|
|
└─/xyz/openbmc_project/inventory/system/chassis/motherboard
|
||
|
|
└─/xyz/openbmc_project/inventory/system/chassis/motherboard/nvme0
|
||
|
|
```
|
||
|
|
|
||
|
|
/xyz/openbmc_project/inventory/system/chassis/motherboard/nvme0 Which
|
||
|
|
implements:
|
||
|
|
|
||
|
|
- xyz.openbmc_project.Inventory.Item
|
||
|
|
- xyz.openbmc_project.Inventory.Decorator.Asset
|
||
|
|
- xyz.openbmc_project.Nvme.Status
|
||
|
|
|
||
|
|
Interface `xyz.openbmc_project.Sensor.Value`, it's for hwmon to monitor
|
||
|
|
temperature and with the following properties:
|
||
|
|
|
||
|
|
| Property | Type | Description |
|
||
|
|
| -------- | ------ | -------------------- |
|
||
|
|
| MaxValue | int64 | Sensor maximum value |
|
||
|
|
| MinValue | int64 | Sensor minimum value |
|
||
|
|
| Scale | int64 | Sensor value scale |
|
||
|
|
| Unit | string | Sensor unit |
|
||
|
|
| Value | int64 | Sensor value |
|
||
|
|
|
||
|
|
Interface `xyz.openbmc_project.Nvme.Status` with the following properties:
|
||
|
|
|
||
|
|
| Property | Type | Description |
|
||
|
|
| ----------------- | ------ | -------------------------------------------- |
|
||
|
|
| SmartWarnings | string | Indicates smart warnings for the state |
|
||
|
|
| StatusFlags | string | Indicates the status of the drives |
|
||
|
|
| DriveLifeUsed | string | A vendor specific estimate of the percentage |
|
||
|
|
| TemperatureFault | bool | If warning type about temperature happened |
|
||
|
|
| BackupdrivesFault | bool | If warning type about backup drives happened |
|
||
|
|
| CapacityFault | bool | If warning type about capacity happened |
|
||
|
|
| DegradesFault | bool | If warning type about degrades happened |
|
||
|
|
| MediaFault | bool | If warning type about media happened |
|
||
|
|
|
||
|
|
Interface `xyz.openbmc_project.Inventory.Item` with the following properties:
|
||
|
|
|
||
|
|
| Property | Type | Description |
|
||
|
|
| ---------- | ------ | ----------------------------------- |
|
||
|
|
| PrettyName | string | The human readable name of the item |
|
||
|
|
| Present | bool | Whether or not the item is present |
|
||
|
|
|
||
|
|
Interface `xyz.openbmc_project.Inventory.Decorator.Asset` with the following
|
||
|
|
properties:
|
||
|
|
|
||
|
|
| Property | Type | Description |
|
||
|
|
| ------------ | ------ | ------------------------------------------------- |
|
||
|
|
| PartNumber | string | The item part number, typically a stocking number |
|
||
|
|
| SerialNumber | string | The item serial number |
|
||
|
|
| Manufacturer | string | The item manufacturer |
|
||
|
|
| BuildDate | bool | The date of item manufacture in YYYYMMDD format |
|
||
|
|
| Model | bool | The model of the item |
|
||
|
|
|
||
|
|
##### xyz.openbmc_project.nvme.manager.service
|
||
|
|
|
||
|
|
This service has several steps:
|
||
|
|
|
||
|
|
1. It will register a D-bus called `xyz.openbmc_project.nvme.manager`
|
||
|
|
description above.
|
||
|
|
2. Obtain the drive index, bus ID, GPIO present pin, power good pin and fault
|
||
|
|
LED object path from the json file mentioned above.
|
||
|
|
3. Each cycle will do following steps:
|
||
|
|
1. Check if the present pin of target drive is true, if true, means drive
|
||
|
|
exists and go to next step. If not, means drive does not exists and remove
|
||
|
|
object path from D-bus by drive index.
|
||
|
|
2. Check if the power good pin of target drive is true, if true means drive
|
||
|
|
is ready then create object path by drive index and go to next step. If
|
||
|
|
not, means drive power abnormal, turn on fault LED and log in journal.
|
||
|
|
3. Send a NVMe-MI command via SMBus Block Read protocol by bus ID of target
|
||
|
|
drive to get data. Data get from NVMe drives are "Status Flags", "SMART
|
||
|
|
Warnings", "Temperature", "Percentage Drive Life Used", "Vendor ID", and
|
||
|
|
"Serial Number".
|
||
|
|
4. The data will be set to the properties in D-bus.
|
||
|
|
|
||
|
|
This service will run automatically and look up NVMe drives every second.
|
||
|
|
|
||
|
|
##### Fault LED
|
||
|
|
|
||
|
|
When the value obtained from the command corresponds to one of the warning
|
||
|
|
types, it will trigger the fault LED of corresponding device and issue events.
|
||
|
|
|
||
|
|
##### Add SEL related to NVMe
|
||
|
|
|
||
|
|
The events `TemperatureFault`, `BackupdrivesFault`, `CapacityFault`,
|
||
|
|
`DegradesFault` and `MediaFault` will be generated for the NVMe errors.
|
||
|
|
|
||
|
|
- Temperature Fault log : when the property `TemperatureFault` set to true
|
||
|
|
- Backupdrives Fault log : when the property `BackupdrivesFault` set to true
|
||
|
|
- Capacity Fault log : when the property `CapacityFault` set to true
|
||
|
|
- Degrades Fault log : when the property `DegradesFault` set to true
|
||
|
|
- Media Fault log: when the property `MediaFault` set to true
|
||
|
|
|
||
|
|
#### Alternatives Considered
|
||
|
|
|
||
|
|
NVMe-MI specification defines multiple commands that can communicate with NVMe
|
||
|
|
drives over MCTP protocol. The NVMe-MI over MCTP has the following key
|
||
|
|
capabilities:
|
||
|
|
|
||
|
|
- Discover drives that are present and learn capabilities of each drives.
|
||
|
|
- Store data about the host environment enabling a Management Controller to
|
||
|
|
query the data later.
|
||
|
|
- A standard format for VPD and defined mechanisms to read/write VPD contents.
|
||
|
|
- Inventorying, configuring and monitoring.
|
||
|
|
|
||
|
|
For monitoring NVMe drives, using NVM Express Basic Management Command over
|
||
|
|
SMBus directly is much simpler than NVMe-MI over MCTP protocol.
|
||
|
|
|
||
|
|
#### Impacts
|
||
|
|
|
||
|
|
This application is monitoring NVMe drives via SMbus and set values to D-bus.
|
||
|
|
The impacts should be small in the system.
|
||
|
|
|
||
|
|
#### Testing
|
||
|
|
|
||
|
|
This implementation is to use NVMe-MI-Basic command over SMBus and then set the
|
||
|
|
response data to D-bus. Testing will send SMBus command to the drives to get the
|
||
|
|
information and compare with the properties in D-bus to make sure they are the
|
||
|
|
same. The testing can be performed on different NVMe drives by different
|
||
|
|
manufacturers. For example: Intel P4500/P4600 and Micron 9200 Max/Pro.
|
||
|
|
|
||
|
|
Unit tests will test by function:
|
||
|
|
|
||
|
|
- It tests the length of responded data is as same as design in the function of
|
||
|
|
getting NVMe information.
|
||
|
|
- It tests the function of setting values to D-bus is as same as design.
|
||
|
|
- It tests the function of turn the corresponding LED ON/OFF by different
|
||
|
|
Smartwarnings values.
|