Hindsight v2.4 Adds JSONL Output

Hindsight v2.4.0 add JSONL output, support for the newest versions of Chrome, and other small fixes. The highlights are:

• Supports Chrome versions 1 - 76
• Adds JSONL output format, which is compatible with Timesketch. The field names in this output type are aligned with Plaso/Timesketch (other output formats remain unchanged).
• Parses other Chrome files, even if History file is absent (as in the case of Time Machine backups)

The SQLite database structures for Chrome 76 (released yesterday) are very similar to past versions. The most notable difference was samesite replacing firstpartyonly in the Cookies database, which I covered in my look at Chromium-based Edge.

JSONL Format Added

JSON Lines (or JSONL) format is similar to JSON, but it's a better fit for potentially large, log-type data than straight JSON. JSONL consists of a single JSON object, typically containing a single "record", on a line. The next line would have another JSON object with the next record, and so on. A JSONL file can be read and processed one line at a time; if the same data was stored in a JSON file, the whole file would need to be read first (which can be problematic with huge timelines).

Here's an example of one line of Hindsight JSONL output, which corresponds to one record/history event:

{"typed_count": 6, "parser": "hindsight/2.4.0", "datetime": "2019-03-12T17:13:42.002825+00:00", "visit_duration": "0:00:21.516643", "visit_count": 13, "message": "https://dfir.blog/chrome-evolution/ (Chrome Evolution) [count: 13]", "data_type": "chrome:history:page_visited", "title": "Chrome Evolution", "source_short": "WEBHIST", "visit_source": "Local", "profile": "/data-sets/chrome/v75", "from_visit": 0, "source_long": "Chrome History", "url_hidden": "false", "transition_friendly": "typed; Navigation Chain End; From Address Bar; Navigation Chain Start; ", "timestamp_desc": "Last Visited Time", "url": "https://dfir.blog/chrome-evolution/", "transition": 838860801, "url_id": 9814}

And here is that same line, but with more whitespace for legibility:

{
  "typed_count": 6,
  "parser": "hindsight/2.4.0",
  "datetime": "2019-03-12T17:13:42.002825+00:00",
  "visit_duration": "0:00:21.516643",
  "visit_count": 13,
  "message": "https://dfir.blog/chrome-evolution/ (Chrome Evolution) [count: 13]",
  "data_type": "chrome:history:page_visited",
  "title": "Chrome Evolution",
  "source_short": "WEBHIST",
  "visit_source": "Local",
  "profile": "/data-sets/chrome/v75",
  "from_visit": 0,
  "source_long": "Chrome History",
  "url_hidden": "false",
  "transition_friendly": "typed; Navigation Chain End; From Address Bar; Navigation Chain Start; ",
  "timestamp_desc": "Last Visited Time",
  "url": "https://dfir.blog/chrome-evolution/",
  "transition": 838860801,
  "url_id": 9814
}

Easier Plaso/Timesketch Integration

Hindsight's JSONL output is meant to make integrating with other tools easier. I use Plaso and Timesketch quite often and now it's much easier to use all three tools together.

Some field names and values in the Hindsight JSONL output are slightly different than in other Hindsight output formats. This is to be more consistent with how Plaso presents browser artifacts and to include fields required by Timesketch. Since Timesketch supports JSONL, now you can upload Hindsight output directly to Timesketch!

The advantage of the field names being consistent with Plaso is that any queries or saved searches in Timesketch will work on browser history events from either tool. It also makes using both Hindsight and Plaso together on a case easier. For example, if you were to first process the browser history from a system with Hindsight, then decide to run the whole disk image through Plaso, you could review the output of both in the same sketch.

Get the New Version

Both the GUI and command line versions of this release are available as:

• compiled exes attached to the GitHub release or in the dist/ folder
• .py versions are available by pip install pyhindsight or downloading/cloning the GitHub repo.