Restore Splunk Frozen Data
“We started off using Splunk’s built-in mechanism to freeze/archive our compliance data. What we didn’t realize at the time was how difficult it would be to restore that data.
Our auditors requested that we go through an exercise to prove we could restore data for a specific time period across specific hosts. That’s when we discovered we had millions of frozen archive files in the S3 archive. Because the entire archive was multiple terabytes of data, we we knew we didn’t have enough disk space to restore all of it, which would have been the easy solution. Our goal was to restore the subset of frozen files requested by the auditors, but we calculated it would take at least 6 person-days to identify the files we needed to restore.
We started searching the web for a solution when we found Conducive and their Archiver for Splunk. Using Conducive’s Archiver we were able to scan and catalog our existing archive, allowing us to restore the exact data requested by the auditors, all in less than 1 day.
We’re now using the Archiver to both manage frozen data archiving, as well as using it to provide reports to the auditors and restore the data as requested. We can do all of this from a UI that lets us choose the date ranges, sourcetypes, indexes and hosts to restore. The entire process usually only takes a few minutes of time.”