Skip to content

Splunk Archiver

Simplify the management of your Splunk data with Conducive's Splunk Archiver. Starting at $2,995.

Document management interface

Why should YOU use Conducive's Archiver for Splunk?

After a year, you will discover that the disk space required for frozen data increases exponentially. 

File management interface

Why does required disk space increase exponentially?

  • Splunk’s default archiving solution doesn’t remove redundant data from buckets or duplicate buckets from a cluster. Splunk freezes all of your data, even though that isn’t necessary.
  • With the following calculation, you can see how easily a 1TB license in a clustered environment (replication factor of 3) can require at least 1 Petabyte of storage every year.

1TB Per Day X 3 (replication factor) X 365 = 1,095 TB per year = 1.095 Petabytes.

Editing a chart

Does the Archiver fix this?

Conducive’s Archiver for Splunk removes redundant data and eliminates duplicate buckets.

Our customers are seeing an average of 80% reduction in storage space per bucket by removing redundant data and an additional 66% reduction by removing duplicate buckets.

For a 1TB license, your annual long-term storage requirement should require less than 100TB per year–a 90% reduction from 1.095 Petabytes.

Frozen File Directory

Can you do this without Conducive's Archiver?

You could modify Splunk’s default script to remove redundant data, but that won’t eliminate duplicate buckets. Also, when you remove the redundant data, you will not be able to track the contents of each bucket when you need to restore data.

You will realize that it's nearly impossible to restore frozen data.

Selecting file from data interface

Why is it difficult to restore data?

  • Splunk’s built-in archiving solution copies frozen buckets to a directory of your choice and leaves everything else up to you. Splunk doesn’t track frozen buckets or help you restore them. You have to manage the storage space, and you have to find the buckets when you want to restore them.
  • A 1TB-per-day clustered environment will create a minimum of 150-300 buckets per day. For 1TB, Splunk will create 36,000-120,000 buckets per year, and in many cases more than that. When you need to restore data, you have to search through each of these buckets (represented as a folder or directory on your file system) to locate the data you want to restore. You also have to ensure you only restore one copy of each bucket. Restoring duplicate buckets could create duplicate search results. Done manually, this could take weeks. Of course, it's possible to write a script to identify some of the data, but that requires time and skills.
Archiver dashboard for restoring data

How does Conducive's Archiver make restoring data easy?

To easily restore frozen data, Conducive’s Archiver tracks all of the details of each bucket, allowing you to restore by host, sourcetype, index, and data range—all with the click of a button.

No more searching through thousands or millions of directories to find the data you want to restore. No more ensuring duplicate buckets aren’t restored. No more hassle. Restore your frozen data with the click of a button.

Case Study

Restore Splunk Frozen Data

We started off using Splunk’s built-in mechanism to freeze/archive our compliance data. What we didn’t realize at the time was how difficult it would be to restore that data. 

Our auditors requested that we go through an exercise to prove we could restore data for a specific time period across specific hosts. That’s when we discovered we had millions of frozen archive files in the S3 archive. Because the entire archive was multiple terabytes of data, we we knew we didn’t have enough disk space to restore all of it, which would have been the easy solution. Our goal was to restore the subset of frozen files requested by the auditors, but we calculated it would take at least 6 person-days to identify the files we needed to restore.

We started searching the web for a solution when we found Conducive and their Archiver for Splunk. Using Conducive’s Archiver we were able to scan and catalog our existing archive, allowing us to restore the exact data requested by the auditors, all in less than 1 day.

We’re now using the Archiver to both manage frozen data archiving, as well as using it to provide reports to the auditors and restore the data as requested. We can do all of this from a UI that lets us choose the date ranges, sourcetypes, indexes and hosts to restore. The entire process usually only takes a few minutes of time.”

Files icon

The Old Way

Search through millions of files to find the frozen data you need to restore.
Optimize Search icon

The New Way

Use Conducive's Archiver to select the date range, source types, hosts, and/or indexes to restore.
Case Study

Archiving Made Simple—IRS Data Retention Requirements

“Due to our work with the IRS, we are required to store 7 years of transactional data for any system that even remotely touches the IRS transactions. With a 200GB license, this translates to about 255TB of uncompressed storage in 7 years. We wanted to store the data securely in the cloud, instead of on-premise. Secondarily, we were worried about restoring frozen data at this volume. We didn’t think it would be easy to find the specific frozen folders that contain the hosts or sources requested by Auditors.”

“We started with the idea of using Splunk’s built-in solution, but wanted a more comprehensive enterprise solution that includes compression, encryption and native cloud storage integration. After talking with a few Splunkers, we were introduced to Conducive and their Archiver for Splunk. Conducive’s solution enabled us to easily manage our Splunk frozen/archived data.”

​After implementing the Archiver, we reduced our storage costs to about $4/TB/month, or a little over $1000 per month in 7 years – and this number is dropping because cloud storage costs are dropping. We’re also able to easily provide reports to auditors and restore data with the click of the mouse.”

Case Study

Auditor Reporting

“Our internal policy requires that we store 6 years of data, and our Auditors have asked that we provide reports proving this data is available and submit to the occasional test to restore the data. We currently keep 18 months of storage accessible. We don’t want to keep 6 years of accessible data on local disk, and we don’t want to use Splunk’s S2 implementation to move searchable data into the cloud. We’d prefer to compress and archive the data to keep the Auditors happy.

We found Conducive’s Archiver for Splunk by searching the Spunk App Store (Splunk Base). Their solution gives us reporting and restoring, along with managed archiving in the cloud. Additionally we can compress and encrypt the data in transit and at rest.

Using Conducive’s Archiver for Splunk, we are able to provide timely reports to our Auditors and restore data as requested.

We’re now using the Archiver to both manage frozen data archiving, as well as using it to provide reports to the auditors and restore the data as requested. We can do all of this from a UI that lets us choose the date ranges, sourcetypes, indexes and hosts to restore. The entire process usually only takes a few minutes of time.”

Features & Benefits

Archiver dashboard for restoring data

One-Click Restore

  • Restore frozen data based on time range, host, sourcetype, and index.
  • Archiver retrieves selected data from storage with the click of a button.
Data shown in Archiver dashboard

Managed Archival Process

  • Ensure that your frozen data is properly archived.
  • Easily create reports for your auditors.
Archiver report dashboard

Reporting

  • Provide auditors with reports proving the data is archived/frozen.
  • Prove to auditors that your data is restored.
  • Have a reporting solution ready when auditors ask.
Data shown in Archiver interface

Data Compression

  • Automatically compress data to about 20% of the original size.
  • Reduce storage costs.
Data shown in Archiver Interface

Encryption

  • Once the data leaves Splunk, it remains encrypted throughout the entire process.
  • Data is encrypted at rest and in flight.
Data shown in Archiver Interface

Deduplicate Data

  • Deduplicate buckets from clustered indexers. (A clustered indexing configuration will contain multiple copies of buckets.)
  • Store only one copy of these buckets, saving time and money.
AWS Archiver Interface

Cloud or On-Premises Storage

  • The Archiver integrates with all major cloud storage providers, or you can choose to store locally.
  • Content as low as $4 per TB per month.
Learn More

Frequently Asked Questions

Why buy from you? What makes you so special?

Conducive has been helping companies derive business value from their data since 2006. We have been been a Splunk partner since 2012. We are a technical company that understands business. Conducive is the only company providing an Enterprise Archiving Solution for Splunk.

Who are you guys, anyway?

Conducive has been in business since 2006. We are a Splunk Accredited PS Partner, a Splunk license re-seller, and technical software developers.

What happens if I wait?

You will be rushed to provide your auditor with reports and you’ll struggle to restore your frozen data.

How much does it cost?

Pricing is based on a number of factors. Schedule a meeting with us so we can assess your situation.

How do I present this to my team?

Tell your team how difficult it is to restore frozen data when you are faced with thousands or millions of frozen buckets, and tell them how the reporting will satisfy your auditor’s requirements.

What does it integrate with?

The Archiver integrates with Splunk as well as all of the major cloud storage provides such as AWS and Google.

Resources

Insights from Our Blog

Conducive's Archiver for Splunk

Conducive's Archiver for Splunk

Watch the video and contact us to schedule a demo.

Read More
5 Tips for Increasing Your Splunk ROI

5 Tips for Increasing Your Splunk ROI

Splunk is a powerful data analytics and visualization tool that can drive serious business results for its users by bringing real-time data insights into every decision in security, IT operations, and so much more.

Read More
Is Splunk a DIY option or do you need a professional partner?

Is Splunk a DIY option or do you need a professional partner?

Splunk has the power to be transformative but only if it’s implemented effectively. So, can you take the DIY route or should you be looking for a qualified Splunk partner?

Read More

Get the most out of your operational intelligence solution today.