Copyright 2015-2020 Hale Landis. All rights reserved.
The goal of NVMEQRWT is to fill up to 15 read/write queues with up to 32 commands in each queue. Error reporting is detailed but some knowledge of an NVME controller's registers and data areas may be needed to fully understand some errors.
NVMEQRWT is a DOS DPMI program. It requires a DPMI driver (the DPMI driver CWSDPMI.EXE is provided with NVMEQRWT).
NVMEQRWT is not shareware or freeware. NVMEQRWT is a commercial product and you or your company must have purchased a copy of the program in order to legally use the full function version of the program. There is a demonstration/evaluation version of the program that has reduced function. The demonstration/evaluation version of NVMEQRWT has no use or distribution restrictions but shall be used only to evaluate the program.
The demonstration/evaluation version of NVMEQRWT has the following restrictions:
NVMEQRWT requires the following basic system and test devices:
NVMEQRWT does not use the system BIOS - NVMEQRWT directly accesses the NVME controller registers and takes full control of the of all NVME data areas and queues. During startup NVMEQRWT resets the NVME controller and creates the queues needed.
Create a bootable DOS device (floppy disk or USB boot device) with no CONFIG.SYS or AUTOEXEC.BAT files. Place the NVMEQRWT.EXE and CWSDPMI.EXE files on the boot device.
Boot your system from the boot device. This insures that you are not loading some strange device drivers that may interfere with NVMEQRWT.
To run NVMEQRWT, set up the test system and devices:
The easy way to start NVMEQRWT is by using the command line
NVMEQRWT
To see a display of NVMEQRWT command line options add the HELP option anywhere in the command line.
When NVMEQRWT starts it will find the NVME controller(s). If more than one NVME controller is found NVMEQRWT will need to be restarted using the b:d:f command line option. During startup NVMEQRWT will reset and initialize the NVME controller. If initialization is successful testing will begin and the screen will switch to "full screen" mode during testing. This screen shows the testing progress. NVMEQRWT also writes a detailed log file.
NVMEQRWT will run until one of the following events stops the program:
The NVMEQRWT command line syntax is:
NVMEQRWT [options]
If no options are specified the program will attempt to test the only NVME controller in the system using all default options.
Except for the HELP and LOG=x option, the command line options are processed left to right. If an option is specified more than once, the last (rightmost) specification is used. Only the first LOG=x option is used, any addtional LOG=x options are ignored.
b:d:f
b:d:f is the PCI bus, device and function for an NVME controller.
When this option is not specified NVMEQRWT will scan PCI buses 0 to 15 for supported NVME host controllers.
This option is not required if there is only one AHCI controller in the system.
If the desired NVME host controller is not found on PCI buses 0 to 15 then this option can be used to specify a PCI bus number up to 255.
BLKSIZE=n
This command line option is ignored starting with version 1A1a.
CMPR=NONE or COMPARE=NONE
CMPR=QUICK or COMPARE=QUICK
CMPR=FULL or COMPARE=FULL
Specify the level of data compare that is done on sectors read. Only sectors that have been written during this execution of NVMEQRWT are subject to data compare when read.
CMPR=NONE or COMPARE=NONE disables all data generation for write commands and all data compare for read commands. This may result in higher IOPS results.
CMPR=QUICK or COMPARE=QUICK compares only the sector tag data and the first and last two bytes of the sectors read (that have been written)..
CMPR=FULL or COMPARE=FULL compares the sector tag data and all of the data bytes of the sectors read (that have been written). This may result in lower IOPS results.
The default is COMPARE=QUICK.
READONLY forces COMPARE=NONE.
Also see: READONLY
DTO=n
Set the NVME Driver TimeOut in seconds. The default is 5 seconds. The valid range of n is 5 to 600 seconds.
LOG=fileName
LOG=*n
LOG=NUL
LOG=NULL
NOTE: Only the first LOG=x option found on the command line is used, any additional LOG=x options are ignored.
The default log file name is ".\NVMEQRWT.LOG". Use this option to specify a different log file name.
LOG=*n can be use to generate unique log file names based on the date/time. The generated file name replaces the *n characters and all following charactes. The LOG=*n options are:
where
LOG=*n examples:
LOG=*1 -- LOG=4E0B47AC.LOG LOG=*2 -- LOG=2C181452.LOG LOG=\logs\*3 -- LOG=\logs\1K25F33B.LOG LOG=dvt\test\*4 -- LOG=dvt\test\E17G4135.LOG
Use LOG=NUL or LOG=NULL to suppress generation of the log file.
MAXERR=n
This option is ignored because NVMEQRWT stops issuing commands when the first error is detected. Additional errors may be reported in the log file because the NVMEQRT waits for all outstanding commands to complete and some of those commands may also report errors.
MAXLBA=nK
MAXLBA=nM
Specify the highest LBA that will be tested. K is 1000 and M is 1000000.
The MINLBA=x and MAXLBA=x options can be used to restrict testing to a small area of a drive.
NOTE: There must be at least 100K sectors in each area, or ( (max_lba-min_lba+1)/num_tags) must be greater than or equal to 100K).
Also see: MINLBA=x
MAXTL=n
Restricts read/write commands to transfer lengths (sector counts) of 1 to 65536. If a device has less than 2000000 sectors, MAXTL for that device may be adjusted so that it does not exceed the value (number_of_sectors / 100).
Also see: MINTL=x
MINLBA=nK
MINLBA=nM
Specify the lowest LBA that will be tested. K is 1000 and M is 1000000.
The MINLBA=x and MAXLBA=x options can be used to restrict testing to a small area of a drive.
NOTE: There must be at least 100K sectors in each area, or ( (max_lba-min_lba+1)/num_tags) must be greater than or equal to 100K).
Also see: MAXLBA=x
MINTL=n
MINTL=*n
MINTL=n restricts read/write commands to transfer lengths (sector counts) of 1 to 65536.
MINTL=*n restricts read/write command to transfer lengths (sector counts of 1 to 65536 and also makes all transfer lengths and LBA values be a multiple of n.
Examples:
Also see: MAXTL=n
MPS=n
Specify the Memory Page Size (MPS). n=4 for 4096 bytes, n=8 for 8192 bytes. The default is 4 (4096 bytes). The device must support the specified MPS.
Please note:
NONSTD=x
Enable or disable program options and/or debug options. x is a list of characters, one character per option. The characters in x may appear in any order.
NOSTOP
This option is ignored because NVMEQRWT stops issuing commands when the first error is detected. Additional errors may be reported in the log file because the NVMEQRT waits for all outstanding commands to complete and some of those commands may also report errors.
NSID=n
Specify the Namespace ID to use (0 to 0xffffffff).
If NSID is not specified it defaults to 1.
The specified NSID must exist before starting NVMEQRWT.
NUMCMD=n
Limit the number of read/write command in each queue. n is 1 to 32. The default is 32.
NUMQUE=n
Limit the number of read/write queues. n is 1 to 15. The default is 15.
Queue 0 is the NVME Admin queue and is initialized at program start.
OKTOWRITE
Use this option to enable writing over user data. When user data is found on the device and this option is used, a prompt is displayed to confirm that it really is OK to write over the user data.
READONLY
No data will be written on the device's media.
The default is WRITEREAD.
SEQ=N
SEQ=WN
SEQ=WRN
SEQ=N specifies normal (random) commands. SEQ=N is the default. Writes are sequential until the the all selected LBA's have been written (logged as SEQ=n). Once all LBA's are written then writes are random (logged as SEQ=N). All reads are random.
SEQ=WN specifies a sequential write pass over the selected LBA's before normal (random) write and read commands are executed..
SEQ=WRN specifies a sequential write pass and a sequential read pass over the selected LBA's before normal (random) write and read commands are executed.
The READONLY option converts
STOP
This option is ignored because NVMEQRWT stops issuing commands when the first error is detected. Additional errors may be reported in the log file because the NVMEQRT waits for all outstanding commands to complete and some of those commands may also report errors.
STOP=x
This option is ignored because NVMEQRWT stops issuing commands when the first error is detected. Additional errors may be reported in the log file because the NVMEQRT waits for all outstanding commands to complete and some of those commands may also report errors.
TIME=n
Specify the run time in minutes. The default is 120 minutes.
Note: There are 1440 minutes in one day, 2880 minutes in two days, etc.
WRITEREAD
Use both write and read commands.
The default is WRITEREAD.
1) Perform a write pass, a read pass and the normal random testing and test for up to 10 hours (600 minutes).
NVMEQRWT SEQ=WRN TIME=600
2) Use the NVME controller on PCI bus 1, device 2 function 0 and test in read only mode.
NVMEQRWT 1:2:0 READONLY
3) Request display of command line options.
NVMEQRWT 1:2:0 READONLY HELP
The command line option defaults are:
COMPARE=QUICK LOG=.\NVMEQRWT.LOG MAXLBA=size_of_device MAXTL=256 MINLBA=0 NSID=1 SEQ=N TIME=120 WRITEREAD
While NVMEQRWT is running, some keys on the keyboard can be used to change what NVMEQRWT is doing. These keys are:
All other keys will restore the full screen display - see FULL SCREEN INFORMATION below.
The NVMEQRWT log file is a plain ASCII text file. This file can be viewed with most text editors and text browsing programs. When printing the file use a font that will allow at least 90 characters per line.
The log file contains the startup configuration messages, the detailed device error messages (if any) and termination summary messages.
Device error messages can contain various levels of information including a command history trace and a low level trace of the ATA I/O port activity.
Once device testing begins, NVMEQRWT enter "full screen" mode. In this mode only summary information and very brief error message data is displayed.
If there are no errors NVMEQRWT places the screen into screen saver mode after 20 minutes of running. If there are no errors pressing the key 's' and 'S' will switch the screen into screen saver mode immediately. Pressing any key, except ESC, 's' or 'S', will restore full screen mode for 20 minutes. Any error will restore full screen mode and disable the screen saver.
In full screen mode 25 lines are displayed:
At startup NVMEQRWT performs the following activities:
Each sector written by NVMEQRWT has the data format shown below. The sector tag data is located at offsets 244 (F4H) to 283 (11BH) in all sectors.
Note that the sector tag fields are in x86 little endian format (least significant byte at lowest byte offset).
byte byte
offset offset length data
decimal hex in bytes pattern
------- ------- -------- ---------------------------------
000-001 000-001 2 a 16-bit incrementing word pattern
002-243 002-0F3 2*121 continuation of the incrementing word pattern
244-247 0F4-0F7 4 ones complement of 248-251
248-251 0F8-0FB 4 RunID (start time)
252-255 0FC-0FF 4 LBA bits 31:0
256-259 100-103 4 LBA bits 63:32 (bits 63:48 are zero)
260-263 104-107 4 ones complement of 264-267
264-267 108-10B 4 command number of the last write
command to this sector
268-271 10C-10F 4 ones complement of 268-271
272-275 110-113 4 sector count of the last write
command to this sector
276-279 114-117 4 bits 31:0 of starting LBA of the
last write command to this sector
280-283 118-11B 4 bits 63:32 of starting lba of the
last write to this to this sector
(bits 63:48 are zero)
284-nnn 11C-xxx 2*n continuation of the incrementing word pattern
The data in bytes 000-002 are called the basic data pattern. This pattern is an incrementing word value used in bytes 2-243 and in bytes 284-nnn.
The data in bytes 244-251 are called the RunID (start time). This field is used to determine if the drive has returned data that was written during the current execution of NVMEQRWT.
The data in bytes 252-259 are called the LBA field. This is the LBA sector address for the sector read.
The data in bytes 260-283 are called the last write command fields. These fields describe the command that performed the last write to this sector. Note that unlike the time and lba fields, NVMEQRWT is not able to know the expected contents of these fields. The command fields are displayed in data compare error messages.
When a sector is read and data compare is done, the fields of the sector are checked in the order shown below. Only sectors with an LBA less than "NextLBA" are compared.
Data compare field checking order:
There is no error recovery for any type of error because NVMEQRWT stops executing commands when the first error is detected.
A typical device I/O error message is:
!!!!!!!!!! I/O ERROR - $curTime$ !!!!!!!!!! CmdNum $cNum$, EC $ec$, EM $emsg$ QN $qNum$, CmdId $cId$ STATUS $status$ - $sMsg$ $cType$ SC $sCnt$, LBA $lba$
The values in the message are:
A typical data compare error message header is:
!!!!!!!!!! DATA COMPARE ERROR - $curTime$ !!!!!!!!!! CmdNum $cNum$, QN $qNum$, CmdId $cId$, Read, SC $sCnt$, LBA $lba$ Sector Offset Description
The header is followed by a description of the error found in each sector read.
$soff$ ------ LBA $lba$ was read - no error found in the data -or- $soff$ ------ LBA $lba$ was read - error found in the data
If the sector's RunId is does not match the current RunID (old or stale data was read) the following is displayed and no additional checking of the sector's data is performed.
$soff$ $boff$ ~RunId exp $edata$ act $adata$ $soff$ $boff$ RunId exp $edata$ act $adata$
If the sector's data was read from the wrong LBA the following is displayed.
$soff$ $boff$ LBA exp $lba$ act $lba$
If the data pattern in the sector (not including the tag data) is incorrect then some number of expected/actual data values are displayed.
$soff$ $boff$ exp $edata$ act $adata$ -and- $soff$ - Only first $n$ of $n$ data errors shown.
For each sector displayed, the last write information is also displayed.
$soff$ - last write CmdNum $cNum$, SC $sCnt$ $soff$ - last write LBA $lba$ -or- $soff$ - last write tag data is not valid
The values in the message are:
The command history trace shows the commands that started and ended up to an including the failing command.
Line BiosTm CmdNum T QN CID CC SC LBA Stat Error nnnn xxxxxx nnnnnnnnnnnnnnn S nn xxxx nn xxx xxxx.xxxx.xxxx.xxxx nnnn xxxxxx nnnnnnnnnnnnnnn E nn xxxx xxxx sss... nnnn xxxxxx nnnnnnnnnnnnnnn * START RESET nnnn xxxxxx nnnnnnnnnnnnnnn * END RESET - STATUS xxxx - ERROR
The values in the trace lines are:
Submission Queue
The queue summary is displayed when NONSTD=F is not used.
---------- Submission Queue Summary ----------
nnn nnn xxxxH xxH xxH xxH xxxxxxxxH xxxxxxxxH xxxxxxxxH xxxxxxxxH
SQN EN CID FUSD PSDT OPC NSID DW10 DW11 DW12
n nn xxxxH xxH xxH xxH xxxxxxxxH xxxxxxxxH xxxxxxxxH xxxxxxxxH
(Next command submission should use entry $qEntry$)
The SQN and EN numbers are decimal. All other values are hex.
A full dump of the queue is displayed when NONSTD=F is used.
---------- Submission Queue Full Dump ---------- Submission Q $qNum$ entry $qEntry$ (next $qEntry$)... DW0-DW3 xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx DW4-DW7 xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx DW8-DW11 xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx DW12-DW15 xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx OpCode $cmdCode$ Fused Op $n$ PSDT $n$ Cmd ID $cId$
The $qNum$ and $qEntry$ values are decimal. All other values are hex.
Completion Queue
The queue summary is displayed when NONSTD=F is not used.
---------- Completion Queue Summary ----------
CQN EN SQID SQHP CID P DW0 STAT NOTE
n nn n nn xxxxH x xxxxxxxxH xxxxH $emsg$
(Next command completion should use entry $qEntry$
The CQN, EN, SQID and SQHP numbers are decimal. $emsg$ describes the STAT (status) value. All other values are hex.
A full dump of the queue is displayed when NONSTD=F is used.
---------- Completion Queue Full Dump ---------- Completion Q $qNum$ entry $qEntry$ (next $qEntry$)... DW0-DW3 xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx SQ ID $qId$ SQ Head Ptr $qEntry$ Status $status$ Phase Bit $pBit$ Cmd ID $cId$
NVMEQRWT issues the follow commands:
During read/write testing only basic read/write commands using sector counts of 1 to 256 are issued on the read/write queues (queues 1 to 15).
Versions not shown were test or skipped versions.
Version 2A1h
Version 2A1g
Version 2A1f
Version 2A1e
Version 2A1d
Version 2A1c
Version 2A1b
Version 2A1a
Version 2A1
Technical support can be found at:
http://www.ata-atapi.com/techsupp.html
-end-