Kismetdb logs
Kismetdb logs
Kismet 2019-04 and newer use a new log format, kismetdb. The kismetdb log unifies device records, location, alerts, console messages, packets, packet-like data, system health snapshots, and more into a single logfile.
Configuring kismetdb
Kismetdb is configured with the other Kismet log types in kismet_logging.conf (or preferably, kismet_site.conf).
For more info on logging check the Kismet docs and for the Kismet site config check the config docs.
Kismetdb logging can be turned on alongside pcapng or ppi logging.
The kismetdb log also provides a number of features while Kismet is running - like the ability to fetch packets surrounding an alert which occurred in the past - so generally you should leave it enabled unless you’ve got very good reasons to turn it off.
What’s a kismetdb, really?
Kismetdb is actually a sqlite3 file - you can read, search, and browse them with any tools that can read sqlite3.
$ sqlite3 foo.kismet
SQLite version 3.27.2 2019-02-25 16:06:06
Enter ".help" for usage hints.
sqlite> .schema
CREATE TABLE KISMET (kismet_version TEXT, db_version INT, db_module TEXT);
CREATE TABLE devices (first_time INT, last_time INT, devkey TEXT, phyname TEXT, devmac TEXT, strongest_signal INT, min_lat REAL, min_lon REAL, max_lat REAL, max_lon REAL, avg_lat REAL, avg_lon REAL, bytes_data INT, type TEXT, device BLOB, UNIQUE(phyname, devmac) ON CONFLICT REPLACE);
CREATE TABLE packets (ts_sec INT, ts_usec INT, phyname TEXT, sourcemac TEXT, destmac TEXT, transmac TEXT, frequency REAL, devkey TEXT, lat REAL, lon REAL, alt REAL, speed REAL, heading REAL, packet_len INT, signal INT, datasource TEXT, dlt INT, packet BLOB, error INT);
CREATE TABLE data (ts_sec INT, ts_usec INT, phyname TEXT, devmac TEXT, lat REAL, lon REAL, alt REAL, speed REAL, heading REAL, datasource TEXT, type TEXT, json BLOB );
CREATE TABLE datasources (uuid TEXT, typestring TEXT, definition TEXT, name TEXT, interface TEXT, json BLOB, UNIQUE(uuid) ON CONFLICT REPLACE);
CREATE TABLE alerts (ts_sec INT, ts_usec INT, phyname TEXT, devmac TEXT, lat REAL, lon REAL, header TEXT, json BLOB );
CREATE TABLE messages (ts_sec INT, lat REAL, lon REAL, msgtype TEXT, message TEXT );
CREATE TABLE snapshots (ts_sec INT, ts_usec INT, lat REAL, lon REAL, snaptype TEXT, json BLOB );
There are a suite of tools included with Kismet to manipulate kismetdb logs, as well, so if talking to SQL directly isn’t your thing, read on!
Statistics and summaries
kismetdb_statistics will summarize a log file and print out some useful stats, like the number of devices, number of packets, datasources, packets seen per datasource, and if GPS is enabled, the bounding rectangle:
$ kismetdb_statistics --in foo.kismet
KismetDB version: 5
Packets: 1373
Non-packet data: 41
Devices: 449
Devices seen between: 2019-04-03 17:14:51 (1554326091) to 2019-04-03 17:35:46 (1554327346)
2 datasources
carnuc-rtl433 rtl433-0 5E600813-0000-0000-0000-00005DBB0805 rtl433
Hardware: Generic RTL2832U OEM
Packets: 41
carnuc-mediatek wlx000e8e5c8866 5FE308BD-0000-0000-0000-000E8E5C8866 linuxwifi
Hardware: mt76x2u
Packets: 1332
Hop rate: 5.000000/second
Hop channels: 1, 1HT40+, 2, 3, 4, 5, 6, 6HT40-, 6HT40+, 7, 8, 9, 10, 11, 11HT40-, 12, 13, 14, 36, 36HT40+, 36VHT80, 40, 40HT40-, 44, 44HT40+, 48, 48HT40-, 52, 52HT40+, 52VHT80, 56, 56HT40-, 60, 60HT40+, 64, 64HT40-, 100, 100HT40+, 100VHT80, 104, 104HT40-, 108, 108HT40+, 112, 112HT40-, 116, 116HT40+, 116VHT80, 120, 120HT40-, 124, 124HT40+, 128, 128HT40-, 132, 132HT40+, 136, 136HT40-, 140, 140HT40-, 149, 149HT40+, 149VHT80, 153, 153HT40-, 157, 157HT40+, 161, 161HT40-, 165, 165HT40-
Bounding location: 40.00000000,-75.000000 41.00000,-75.000000 (~19.390973 Km)
Packets with location: 1373
Data with location: 41
It can also out put JSON records with the --json flag; if you’re indexing your logs, the JSON output gives you formatted data:
$ kismetdb_statistics --in foo.kismet --json
{
"data_packets" : 41,
"datasources" :
[
{
"definition" : "rtl433-0:name=carnuc-rtl433",
"hardware" : "Generic RTL2832U OEM",
"interface" : "rtl433-0",
"name" : "carnuc-rtl433",
"packets" : 41,
"type" : "rtl433",
"uuid" : "5E600813-0000-0000-0000-00005DBB0805"
},
{
"definition" : "wlx000e8e5c8866:name=carnuc-mediatek",
"hardware" : "mt76x2u",
"hop_channels" : "1, 1HT40+, 2, 3, 4, 5, 6, 6HT40-, 6HT40+, 7, 8, 9, 10, 11, 11HT40-, 12, 13, 14, 36, 36HT40+, 36VHT80, 40, 40HT40-, 44, 44HT40+, 48, 48HT40-, 52, 52HT40+, 52VHT80, 56, 56HT40-, 60, 60HT40+, 64, 64HT40-, 100, 100HT40+, 100VHT80, 104, 104HT40-, 108, 108HT40+, 112, 112HT40-, 116, 116HT40+, 116VHT80, 120, 120HT40-, 124, 124HT40+, 128, 128HT40-, 132, 132HT40+, 136, 136HT40-, 140, 140HT40-, 149, 149HT40+, 149VHT80, 153, 153HT40-, 157, 157HT40+, 161, 161HT40-, 165, 165HT40-",
"hop_rate" : 5.0,
"interface" : "wlx000e8e5c8866",
"name" : "carnuc-mediatek",
"packets" : 1332,
"type" : "linuxwifi",
"uuid" : "5FE308BD-0000-0000-0000-000E8E5C8866"
}
],
"device_max_time" : 1554327346,
"device_min_time" : 1554326091,
"devices" : 449,
"diag_distance_km" : 19.39097317382431,
"file" : "carnuc-20190403-17-14-49-1.kismet",
"kismetdb_version" : 5,
"max_lat" : 40.000000,
"max_lon" : -75.000000,
"min_lat" : 41.000000,
"min_lon" : -76.0000000,
"packets" : 1373
}
There’s more info on the kismetdb_statistics tool in the Kismet docs.
Uploading to Wigle
Converting a kismetdb log to a Wigle-formatted CSV is also trivial, using the kismetdb_to_wiglecsv tool:
$ kismetdb_to_wiglecsv --in foo.kismet --out foo.wigle --verbose
* Found KismetDB version 5
* Found 449 devices, 1256 usable packets, 1373 total packets
* Starting to process file, max device cache 1000
* 5%% processed 62 records, 0 discarded from rate limiting, 0 discarded from exclusion zones, 25 cached
* 10%% processed 124 records, 0 discarded from rate limiting, 0 discarded from exclusion zones, 28 cached
* 15%% processed 186 records, 0 discarded from rate limiting, 0 discarded from exclusion zones, 29 cached
* 20%% processed 248 records, 0 discarded from rate limiting, 0 discarded from exclusion zones, 62 cached
* 25%% processed 310 records, 0 discarded from rate limiting, 0 discarded from exclusion zones, 90 cached
* 30%% processed 372 records, 0 discarded from rate limiting, 0 discarded from exclusion zones, 112 cached
* 35%% processed 434 records, 0 discarded from rate limiting, 0 discarded from exclusion zones, 146 cached
* 40%% processed 496 records, 0 discarded from rate limiting, 0 discarded from exclusion zones, 158 cached
* 45%% processed 558 records, 0 discarded from rate limiting, 0 discarded from exclusion zones, 184 cached
* 50%% processed 620 records, 0 discarded from rate limiting, 0 discarded from exclusion zones, 194 cached
* 55%% processed 682 records, 0 discarded from rate limiting, 0 discarded from exclusion zones, 223 cached
* 60%% processed 744 records, 0 discarded from rate limiting, 0 discarded from exclusion zones, 255 cached
* 65%% processed 806 records, 0 discarded from rate limiting, 0 discarded from exclusion zones, 271 cached
* 70%% processed 868 records, 0 discarded from rate limiting, 0 discarded from exclusion zones, 284 cached
* 75%% processed 930 records, 0 discarded from rate limiting, 0 discarded from exclusion zones, 307 cached
* 79%% processed 992 records, 0 discarded from rate limiting, 0 discarded from exclusion zones, 322 cached
* 84%% processed 1054 records, 0 discarded from rate limiting, 0 discarded from exclusion zones, 339 cached
* 89%% processed 1116 records, 0 discarded from rate limiting, 0 discarded from exclusion zones, 353 cached
* 94%% processed 1178 records, 0 discarded from rate limiting, 0 discarded from exclusion zones, 365 cached
* 99%% processed 1240 records, 0 discarded from rate limiting, 0 discarded from exclusion zones, 383 cached
* Processed 1256 records, 0 discarded from rate limiting, 0 discarded from exclusion zones
* Done!
Output can be rate-limited for very large files, and regions (like your house or office) can be excluded.
More info on the wiglecsv tool is in the Kismet docs.
Dumping devices and stripping packets
The JSON records of all the devices seen by Kismet can be exported with the kismetdb_dump_devices tool; this is most useful when combining with other tools or scripts to index devices seen by Kismet.
Sometimes you don’t need (or want) to keep the packets in a kismetdb log; you can remove them with the kismetdb_strip_packets tool.
You can read more about dumping devices and stripping packets in the docs.
Exporting to pcap
Exporting to pcap is done with the kismetdb Python module, which is available via pip:
$ pip install kismetdb
Pcap files can then be made with the kismet_log_to_pcap tool:
$ kismet_log_to_pcap --in foo.kismet --out foo.pcap
Packets can be filtered by signal strength, capture datasource, time frame, and associated devices, too.
Click here for more information from the Kismet docs.
Working with kismetdb directly
The Kismetdb schema is documented in the developer docs. Each table typically contains some normalized searchable data (location, signal, source IDs, and similar), as well as a JSON record of the complete encoded data.
It is possible, but not generally recommended, to open a kismetdb file while Kismet is running. Generally, it’s much better to use the live APIs for accessing data - if you performed a long operation on a live kismetdb log, it might stall (or even kill) the Kismet server accessing it, and some data is only written at intervals to save processing time. The live APIs will always return the latest device records, etc, and the historic APIs allow access to much of the data in the kismetdb log via the REST system.