Defining a Field for a User Record
Comment Lines start with a '!' character
A User Record Definition should contain at least two Field entries. Define each field using the following syntax:
! All Records must have this Field as the first entry
FIELD
ARRAY_OCCURS
TYPE BINARY_32
OUT_LEN 0
! This is the standard field definition, the tag START_ARRAY, must be used only once and this should be placed on the second Field entry
FIELD
START_ARRAY
NAME [<language-id>] <short_field-name>
LONG_NAME "<long_field_name>"
HEAD [<language-id>] "<field-heading>"
DESC [<language-id>] "<field-description>"
OUT_LEN <zero or non-zero value>
TYPE {BINARY_16 [,<decimal-places>] | BINARY_32 [,<decimal-places>] | STRING <length>}
DISPLAY_OPTIONS {ALARM | ELAPSED_TIME | FILENAME | GRAPH | PROCESS | RESPONSE_TIME | RESP_TIME_HOURS | SORT | TIMESTAMP | USER | WEIGHTED_VALUE}
DB_OPTION {ACCUMULATE | AVERAGE | AVERAGE_OVER_INT | MAXIMUM | MINIMUM | REPLACE}
[OPTION CUSTOMER PII DEVICE COMPONENT]
! Repeat Field entries for all the required data to capture in the Record
FIELD
NAME [<language-id>] <short_field-name>
LONG_NAME "<long_field_name>"
...
Syntax Elements
FIELD
The FIELD token indicates the start of each field definition.
The first fields, up to the START_ARRAY entry are 'static' fields. Static fields should generally be avoided unless they are being used as associate fields. Even then, it is best to provide a copy of these fields in the START_ARRAY section to allow for their use in where clause filtering.
ARRAY_OCCURS
This is a special token which is used to indicate the static field that will contain the number of array instances in any given data delivery from a collection. There must always be one of these static fields and it must always be specified as shown in the example below. It does not have to be the first static field. It does not get a name and it is always of type binary_32.
Example;
FIELD ARRAY_OCCURS TYPE BINARY_32 OUT_LEN 0
START_ARRAY
This token identifies the first field of the array and indicates the end of the 'static' field entries. It also indicates the start of the key fields. START_ARRAY and all fields after START_ARRAY are considered part of the array.
This token must be included once in the record definition otherwise all fields will be considered to be static fields.
BEWARE: A common mistake is to accidentally delete the START_ARRAY clause while cutting and pasting fields. The result is still syntactically valid. It just specifies a record entirely made up of static fields. Probably not what was expected and not particularly useful.
Example;
FIELD ARRAY_OCCURS TYPE BINARY_32 OUT_LEN 0 FIELD START_ARRAY NAME ENG MEM_FREE LONG_NAME "MemoryFreeMB" HEAD ENG "Mem Free" DESC ENG "Amount of free physical memory on the machine in MB"
NAME [<language-id>]
This is the field ‘short’ name which all fields must have. Within Prognosis the ‘short’ name is not particularly visible, but they are still required for use in where-clauses and in the syntax for Prognosis Analyst scripts and so must be defined. The <language-id> function is not currently implemented and can be omitted.
The 'short' field name:
Can contain a maximum of 8 characters.
Must be unique within its own record definition. However, the short and long names for the same field may be identical.
A field 'short' name must not be changed after a record has been in use. Changing the short name of a field may invalidate databases and any Windows Client documents that reference it (sometimes such references represent implicit rather than explicitly selected fields).
LONG_NAME
The LONG_NAME is the name commonly seen in the Windows Client for selection in the configuration of Displays and other uses of records. This name may be quite long, but for practical purposes (fitting in dialogs etc), it is best to keep it under 32 characters.
The 'long' field name:
Must be contained within quotes.
Must be unique within its own record definition. However, the short and long names for the same field may be identical.
The field LONG_NAME can be changed if required, the only limitation being the potential confusion caused to existing users who may be familiar with the previous name.
HEAD [<language-id>]
This heading text provides a default column heading for tabular displays. However, its actual practical usage is more limited than it may first appear. It was originally provided for use on block mode terminal screens, where screen real estate was at a premium.
The field heading must be enclosed in quotation marks and can be up to 8 characters long. Field headings are displayed as two rows of 4 characters, one on top of the other.
Example
HEAD ENG "Mem Free"
The result of this entry would actually appear as:
Mem Free
This may still be useful for some reports, but most users that are trying to provide a well laid out and informative/readable Display or report will override these default headings by using the facilities provided in the Windows Client Display configuration.
The <language-id> function is not currently implemented and can be omitted.
DESC [<language-id>]
This is the field description, which is displayed in the tooltips when selecting records and fields or when hovering over tabular display headings in the Windows Client. The <language-id> function is not currently implemented and can be omitted.
The description is enclosed in quotations and can be up to 32,767 characters long. However, in practice, more than 20 or so lines will be of limited value in the user interface. In the tooltips, if the lines of the record description plus the lines of field description exceed the screen size, then they will be truncated and there is no way to scroll tooltips
TYPE
This token is used to specify the actual field format.
BINARY_16 | A signed 16-bit integer value (-32768 to 32767). Generally, avoid the use of 16-bit values, unless there is a 100% certainty that it will never overflow. This can be difficult, for example, if more than 327 percentage fields are ever accumulated in a Display COMBINE operation then it may overflow. |
BINARY_32 | A signed 32-bit integer value (-2147483648 to 2147483647). |
BINARY_64 | A signed 64-bit integer value (-9223372036854775808 to 9223372036854775807). |
STRING <length> | An ASCII character string of <length> characters. The length must be a multiple of 2 bytes and less than 255. All string fields must be space padded by the collection and should never contain control characters such as line feeds, carriage return or NULLs. |
A field can have decimal precision. You specify decimal precision by appending the number of decimal places to the end of the TYPE separated by a comma (,). Only binary fields can have decimal places.
Example:
Field type “BINARY_16, 2” is a signed 16-bit integer value with an implicit or virtual decimal place (-327.68 to 327.67)
or
Field type “BINARY_32, 4” is a signed 32-bit integer value with 4 implicit or virtual decimal places (-214748.3648 to 214748.3647).
or,
Field type “BINARY_64, 6” is a signed 64-bit integer value with 6 implicit or virtual decimal places (-9223372036854.775808 to 9223372036854.775807).
Where-clause numeric processing is always performed in a “Binary64, 4” format. Therefore, where-clauses that try to filter on values with more than 4 decimal places may lose accuracy.
Percentages are most commonly represented as Binary -32,2.
OUT_LEN
This token is now mostly redundant. As with the HEAD (field heading) token, it was originally used to represent the column width needed on a block mode terminal screen to display the field.
UNIX, Windows and more recent versions of HPE NonStop will all ignore this token. They provide an initial default layout, based on the data seen in a column and allow the user to create persistent adjustments using their mouse.
OUT_LEN 0 has a special meaning, it causes the field to be hidden from users in the user interface. This is the field equivalent of the record OPTIONS SUPPRESS token. Non-zero values make the field visible, however they have no effect on the presented length of the field.
DISPLAY_OPTIONS
This token is used mostly to specify how the field should be formatted for display. However, in some cases, it can also be used to modify the internal layout of the field.
The following options are available. However, not all DISPLAY_OPTIONs make sense for all TYPE (field type) tokens.
ALARM | This option is obsolete. |
ELAPSED_TIME | Only applies to BINARY_32 field types. This option specifies that the field is to be interpreted as an elapsed time in units of seconds. It is display formatted as <Days>.<hh>:<mm>:<ss> with leading zeroes suppressed. For example, 6.18:25:23 represents 6 days, 18 hours, 25 minutes and 23 seconds. |
FILENAME | Not applicable for Windows or UNIX systems. |
GRAPH | This option is obsolete. |
PROCESS | Not applicable for Windows or UNIX systems. |
RESPONSE_TIME | Only applies to BINARY_64 field types. This option specifies that the field is internally formatted as a 32-bit transaction counter, followed by a 64-bit microsecond quantity of time. Therefore, it is 12 bytes long in total. The intended use is for recording transaction response times. The 64-bit microsecond time is the total amount of time measured in all transactions counted. The counter is the count of transactions measured. On presentation, the time is divided by the count and presented like an elapsed time field. The reason for representing response time fields in this way is that they really represent a weighted average of the response time. Whenever Prognosis performs aggregation operations such as database summary or Windows Client combine operations, it must account for the number of transactions in each sample to provide a correct result. |
RESP_TIME_HOURS | This is in most regards, identical to the RESPONSE_TIME option. The only functional difference is that it is displayed as an elapsed time in hours rather than seconds. This is because its target use is for response times that are typically expected to be quite long, such as the Mean Time Between Failure (MTBF) of modern hardware. |
SORT | This option is now obsolete as all field types are sortable. |
TIMESTAMP | Only applies to BINARY_64 field types. This option specifies that the field is to be interpreted as a timestamp. It is internally formatted as a signed 64-bit microsecond resolution timestamp field, followed by a signed 32-bit GMT offset in seconds. Therefore, 12 bytes long in total. The base for the timestamp is 12:00 Jan 01, 4713BC. This is the native timestamp format on HP Nonstop systems and is used throughout Prognosis. Conversion from a Windows file time is: GetSystemTimeAsFileTime((LPFILETIME)&llResult); // UTC 100 NanoSecs since start 1601 llResult /= 10; // convert to microseconds llResult += 199222286400000000; // convert to microseconds // since 12:00 Jan 01, 4713BC. Conversion from UNIX gettimeofday() is: gettimeofday(&tv, &tz); timestamp=((XLongLong) tv.tv_sec * LL(1000000)) +((XLongLong) tv.tv_usec) + (XLongLong)210866760000000000 // usec between 1/1/4713BC 12.00 and 1/1/1970 00.00 Special note for Windows Collection developers: Be careful when using timestamp fields as key fields to uniquely identify instances such as events. The system time on Windows can be obtained by using calls such as GetSystemTimeAsFileTime() continuously, in a hard loop, and the resulting timestamp will only change approximately every 20th of a second. You may want to define an additional qualifying key field to force uniqueness, e.g. an event sequence number. Special display formatting options are allowed in the Prognosis user interfaces to allow a great variety of timestamp presentations. |
USER | The field contains a numeric user ID and will be displayed in the form GROUP, USER. The field must be TYPE BINARY_16. |
WEIGHTED_VALUE | This is, in most regards, identical to the RESPONSE_TIME option. The only functional difference is that it is displayed as a regular number. Also, the number of decimal places is relevant in these fields. The ‘Value’ part of the field should be stored in the 64-bit number, with the defined number of decimal places and it will also be displayed with that number of decimal places. |
DB_OPTION
The DB_OPTION token can be very useful and is also the attribute most often specified incorrectly. To understand its purpose, it is necessary to consider the interval or time based nature of Prognosis data.
Prognosis records are collected at regular intervals. Statistical data provided in a collection such as rates, deltas or percentage busy fields are related to the size of the interval on which they were gathered.
DB_OPTION specifies the aggregation operation to apply when summarizing Prognosis data up from short intervals to longer intervals. These will be applied automatically if data is replayed from a Prognosis database with a larger replay interval than the interval on which the data was originally collected into the database. Rows from each of the original shorter interval samples will be matched, based on matching keys, and their fields aggregated to produce data for the same instances, summarized to the larger interval. This also enables the database replay zoom in/out feature in the Prognosis user interface.
As a general rule, choose the DB_OPTION aggregation that would produce the same answer as if the data had been originally gathered on the longer interval.
Valid DB_OPTIONs are:
ACCUMULATE | Add fields together, e.g. a TransactionCount field might use this option. The total of the transaction counts from each of 6 successive 10 second collections would be the same as if the transaction count had been recorded over a one minute collection interval. |
AVERAGE | Average of the actual data instances seen on the shorter intervals, e.g. a TransactionResponseTime field might use this option. A summary record, showing the average response time for each transaction type, would need to have its response times averaged over several intervals to produce the longer interval result. Most importantly, if one or more of the original collection intervals did not have a row for a particular transaction type (because there weren’t any), then it would not be appropriate for those intervals to affect the average. |
AVERAGE_OVER_INT | Average of the data instances seen over all the shorter intervals, e.g. a process CpuBusy% field might use this option. This is different from AVERAGE, in that it counts all of the original collection intervals in the average calculation, regardless of the presence or absence of the data instance in each collection interval. A process, being a transient entity, may be present in all of the original collection intervals being summarized up to the larger interval. However, the desired result would be to show the CpuBusy% utilization as a proportion of the whole of the larger interval, as it would have been represented if originally collected on the larger interval. |
MAXIMUM | Select the largest value. This option should be used where the field can contain some type of maximum value. |
MINIMUM | Select the smallest value. This option should be used where the field can contain some type of minimum value. |
REPLACE | Use the value from the last interval in which each instance was represented. Typically, string fields always have this option. There are numeric fields where this makes sense. For example, the Handles field of the Windows NtProcess record uses DB_OPTION REPLACE because the number of handles open at the time of collection is the correct answer. Therefore, taking the last interval value gives the same result as if the data was originally gathered on a longer interval. |
| ONLINE_ONLY | The field cannot be collected in a database. |
OPTION
This token is optional and used to specify various behaviors for the field.
| DEVICE -New- | Adding this token to a STRING field means the value of the field will be used as the 'Source Node' for alerts raised when a threshold is triggered for the record. This is useful for Prognosis records collected for systems that do not register Prognosis nodes, such as Session Border Controllers and Gateways devices. When the option is not specified in a Prognosis record, alerts raised for these records will continue to set the 'Source Node' as the Prognosis Monitoring node name. This token should only be applied to a single field in the Record. It is possible that in some pre-packaged Thresholds already have specified the device field in the alert message. Therefore adding the DEVICE token to this field will result in the name being used twice in the Threshold alert message. For example, see Threshold - IptDeviceTh The value of the field having this token will also be used in the context of Alert Suppression to identify a Device (in a record with 'OPTIONS DEVICE' token) or link to a Device or Prognosis node (in a record with 'OPTIONS COMPONENT' and 'REFERENCES' tokens). |
| COMPONENT -New- | Adding this token to a STRING field means the value of the field can be used to identify individual components in alerts raised when a threshold is triggered for the record. This token only has meaning for records having 'REFERENCES' and 'OPTIONS COMPONENT' record level tokens. It is part of the Alert Suppression feature in Prognosis Management nodes. For records using 'REFERENCES PrognosisNode', the Component this row represents is logically attached to Prognosis node (such as computer running Prognosis or server that Prognosis monitors off-board). Records specifying a name other than 'PrognosisNode' in the 'REFERENCES' token are Components of the Device represented by the record specified in the 'REFERENCES' token. For records using 'REFERENCES' token without this option on a field, the value of the meta field NODE is used instead. Using both DEVICE and COMPONENT options on the same field defines one-to-one relationship between this extension record and the primary record for the Device. |
| CUSTOMER | Adding this token to a field will mark it as the field to check when conducting 'customer based filtering' which is a feature of the Security Role function in the Prognosis Web Application. Only one field in the Record can be marked with this token |
| PII | This specifies the field as containing Personal Identifiable Information (PII). Prognosis will use this tag to obfuscate the data when it is stored into any database. This token can be applied to many fields in the record as required. |