Query Details and Stats
Each HTTP query can include some of these details such as the original query and the timing information using the and showQuery
parameters. Other details can be found in the /api/stats/query
output including headers, status and exceptions. And full details (minus the actual result data) can be logged to disk via the logging config. This page details the various query sections and the information found therein.
This section is a serialization of the query given by the user. In the logs and stats page this will be the full query with timing and global options. When returned with the query results, only the sub query (metric and filters) are returned with the associated result set for identification purposes (e.g. if you request the same metric twice with two different aggregators, you need to know which data set corresponds to which aggregator).
For the fields and what they mean, see /api/query. Some notes about the fields:
The
tags
map should have the same number of entries as the array hasgroup_by
entries. This is due to backwards compatibility with 2.1 and 1.0. Old style queries are converted into filtered queries and function the same way.A number of extra fields may be shown here with their default values such as
null
.You can copy and paste the query into a POST client to execute and find out what data was returned.
Exception
If the query failed, this field will include the message string and the first line of the stack trace for pinpointing. If the query was successful, this field will be null.
For future use, this field can be used to extract user information from queries and help debug who is using a TSD the most. It’s fairly easy to modify the TSD code to extract the user from an HTTP header.
RequestHeaders
This is a map of the headers sent with the HTTP request. In a mediocre effort at security, the Cookie
header field is obfuscated with asterisks in the case that it contains user identifiable or secure information. Each request is different so lookup the headers in the HTTP RFCs or your web browser or clients documentation.
This field contains the numeric HTTP response code and a textual representation of that code.
Other Fields
The output for log files and the stats page include other fields with single values as listed below:
A number of statistics are available around each query and more will be added over time. Various levels of detail are measured including:
Global - Metrics pertaining to the entire query including max and average timings of each sub query.
Per-Sub Query - Metrics pertaining to a single sub query (if multiple are present) including max and average timings of scanner.
Global stats are printed to the standard log, stats page. The full global, sub query and scanner details are available in the query log and via the query API when is present. Timing stats at a lower level are aggregated into max and average values at the upper level. Counters at each lower level are also aggregated at each upper level so you’ll see the same counter metrics at each level. A table of stats and sections appears below.
Note
Metric | Type | Section | Description |
---|---|---|---|
compactionTime | Float | Scanner | Cumulative time spent running each row through the compaction code to create a single column and manage duplicate values. |
hbaseTime | Float | Scanner | Cumulative time spent waiting on HBase to return data. (Includes AsyncHBase deserialization time). |
scannerId | String | Scanner | Details about the scanner including the table, start and end keys as well as filters used. |
scannerTime | Float | Scanner | The total time from initialization of the scanner to when the scanner completed and closed. |
scannerUidToStringTime | Float | Scanner | Cumulative time spent resolving UIDs from row keys to strings for use with regex and wildcard filters. If neither filter is used this value should be zero. |
successfulScan | Integer | Scanner, Query, Global | How many scanners completed successfully. Per query, this should be equal to the number of salting buckets, or |
uidPairsResolved | Integer | Scanner | Total number of row key UIDs resolved to tag values when a regex or wildcard filter is used. If neither filter is used this value should be zero. |
aggregationTime | Float | Query | |
groupByTime | Float | Query | Cumulative time spent sorting scanner results into groups for future aggregation. |
queryScanTime | Float | Query | Total time spent waiting on the scanners to return results. This includes the |
saltScannerMergeTime | Float | Query | Total time spent merging the salt scanner results into a single set prior to group by operations. |
serializationTime | Float | Query | Total time spent serializing the query results. This includes |
uidToStringTime | Float | Query | Cumulative time spent resolving UIDs to strings for serialization. |
emittedDPs | Integer | Query, Global | The total number of data points serialized in the output. Note that this may include NaNs or Nulls if the query specified such. |
queryIndex | Integer | Query | The index of the sub query in the original user supplied query list. |
processingPreWriteTime | Float | Global | Total time spent processing, fetching data and serializing results for the query until it is written over the wire. This value is sent in the API summary results and used as an estimate of the total time spent processing by the TSD. However it does not include the amount of time it took to send the value over the wire. |
totalTime | Float | Global |