130 Clients and Counting
Why should YOU use Conducive's Archiver for Splunk?
AFTER A YEAR, YOU WILL DISCOVER THAT THE DISK SPACE REQUIRED FOR FROZEN DATA INCREASES EXPONENTIALLY
Why does required disk space increase exponentially?
- 1Splunk's default archiving solution doesn't remove redundant data from buckets or duplicate buckets from a cluster. Splunk freezes all of your data, even thought that isn't necessary.
- 2With the following calculation you can see how easily a 1TB license in a clustered environment (replication factor of 3) can require at least 1 Petabyte of storage every year.
1TB Per Day X 3 (replication factor) X 365 = 1,095 TB per year = 1.095 Petabytes.
Does the Archiver fix this?
Conducive's Archiver for Splunk removes redundant data and eliminates duplicate buckets.
Our customers are seeing an average of 80% reduction in storage space per bucket by removing redundant data and an additional 66% reduction by removing duplicate buckets.
For a 1TB license, your annual long-term storage requirement should require less than 100TB per year - a 90% reduction from 1.095 Petabytes.
Can you do this without Conducive's Archiver?
You could modify Splunk's default script to remove redundant data, but that won't eliminate duplicate buckets. And, when you remove the redundant data you will not be able to track the contents of each bucket when you need to restore data.
YOU WILL REALIZE THAT IT'S NEARLY IMPOSSIBLE TO RESTORE FROZEN DATA
Why is it difficult to restore data?
- 1Splunk's built-in archiving solution copies frozen buckets to a directory of your choice and leaves everything else up to you. Splunk doesn't track frozen buckets or help you restore them. You have to manage the storage space, and you have to find the buckets when you want to restore them.
- 2A 1TB per day clustered environment will create a minimum of 150 to 300 buckets per day. For 1TB, Splunk will create between 36,000 and 120,000 buckets per year, and in many cases more than that. When you need to restore data, you have to search through each of these buckets (represented as a folder or directory on your file system) to locate the data you want to restore. You also have to ensure you only restore one copy of each bucket. Restoring duplicate buckets could create duplicate search results. Done manually, this could take weeks. Of course, its possible to write a script to identify some of the data, but that requires time and skills.
How does Conducive's Archiver make restoring data easy?
To easily restore frozen data, Conducive's Archiver tracks all of the details of each bucket, allowing you to restore by host, sourcetype, index and data range - all with the click of a button.
No more searching through thousands or millions of directories to find the data you want to restore. No more ensuring duplicate buckets aren't restored. No more hassle. Restore your frozen data at the click of a button.
Case Studies
Restore Splunk Frozen Data
"We started off using Splunk's built-in mechanism to freeze/archive our compliance data. What we didn't realize at the time was how difficult it would be to restore that data.
Our auditors requested that we go through an exercise to prove we could restore data for a specific time period across specific hosts. That's when we discovered we had millions of frozen archive files in the S3 archive. Because the entire archive was multiple terabytes of data, we we knew we didn't have enough disk space to restore all of it, which would have been the easy solution. Our goal was to restore the subset of frozen files requested by the auditors, but we calculated it would take at least 6 person-days to identify the files we needed to restore.
We started searching the web for a solution when we found Conducive and their Archiver for Splunk. Using Conducive's Archiver we were able to scan and catalog our existing archive, allowing us to restore the exact data requested by the auditors, all in less than 1 day.
We're now using the Archiver to both manage frozen data archiving, as well as using it to provide reports to the auditors and restore the data as requested. We can do all of this from a UI that lets us choose the date ranges, sourcetypes, indexes and hosts to restore. The entire process usually only takes a few minutes of time."
- National Retailer located in the Midwest
OLD WAY: Search through millions of files to find the frozen data you need to restore.
Directory of frozen data.
NEW WAY: Restore at the click of a button.
Use Conducive's Archiver to select the date range, source types, hosts and/or indexes to restore.
Click a button to restore.
- Government Agency
Archiving Made Simple - IRS Data Retention Requirements
"Due to our work with the IRS, we are required to store 7 years of transactional data for any system that even remotely touches the IRS transactions. With a 200GB license, this translates to about 255TB of uncompressed storage in 7 years. We wanted to store the data securely in the cloud, instead of on-premise. Secondarily, we were worried about restoring frozen data at this volume. We didn't think it would be easy to find the specific frozen folders that contain the hosts or sources requested by Auditors."
Auditor Report: Source Types and Hosts
"We started with the idea of using Splunk's built-in solution, but wanted a more comprehensive enterprise solution that includes compression, encryption and native cloud storage integration. After talking with a few Splunkers, we were introduced to Conducive and their Archiver for Splunk. Conducive's solution enabled us to easily manage our Splunk frozen/archived data."
After implementing the Archiver, we reduced our storage costs to about $4/TB/month, or a little over $1000 per month in 7 years - and this number is dropping because cloud storage costs are dropping. We're also able to easily provide reports to auditors and restore data with the click of the mouse."
Auditor Reporting
"Our internal policy requires that we store 6 years of data, and our Auditors have asked that we provide reports proving this data is available and submit to the occasional test to restore the data. We currently keep 18 months of storage accessible. We don't want to keep 6 years of accessible data on local disk, and we don't want to use Splunk's S2 implementation to move searchable data into the cloud. We'd prefer to compress and archive the data to keep the Auditors happy.
We found Conducive's Archiver for Splunk by searching the Spunk App Store (Splunk Base). Their solution gives us reporting and restoring, along with managed archiving in the cloud. Additionally we can compress and encrypt the data in transit and at rest.
Using Conducive's Archiver for Splunk, we are able to provide timely reports to our Auditors and restore data as requested.
We're now using the Archiver to both manage frozen data archiving, as well as using it to provide reports to the auditors and restore the data as requested. We can do all of this from a UI that lets us choose the date ranges, sourcetypes, indexes and hosts to restore. The entire process usually only takes a few minutes of time."
- Midsized Manufacturer based in the Midwest
Features and Benefits
One click restore .
Managed archival process
Reporting
4
Compress data to save storage space
Encryption.
Deduplicate data.
Cloud or on-premise storage
Questions and Answers
Why buy from you? | Conducive has been helping companies derive business value from their data since 2006. We have been been a Splunk partner since 2012. We are a technical company that understands business. Conducive is the only company providing an Enterprise Archiving Solution for Splunk. |
|---|---|
Who are you guys, anyway? | Conducive has been in business since 2006. We are a Splunk Accredited PS Partner, a Splunk license reseller and technical software developers. |
What happens if I wait? | You will be rushed to provide your auditor with reports and you'll struggle to restore your frozen data. |
How much does it cost? | The price is based on a number of factors. Schedule a meeting with us so we can assess your situation. |
Are there any long term contracts? | We require a minimum 3 year contract. |
How do I present this to my team? | Tell your team how difficult it is to restore frozen data when you are faced with thousands or millions of frozen buckets, and tell them how the reporting will satisfy your auditor's requirements. |
What does it integrate with? | The Archiver integrates with Splunk as well as all of the major cloud storage provides such as AWS and Google. |
The Team
Founder and CEO of Conducive
Architecting Splunk since 2012.
Solving business data issues since 1997.
Randy Hammelman
Certified Splunk Architect since 2012.
Co-author of Building Splunk Solutions .conf2015 edition.
Makes Splunk function optimally.
Brian Schutz
Certified Splunk Architect since 2016.
Develops solutions for large companies with process and longevity in mind.
Solves all of the problems in Splunk.
Andres Banuelos
30 Day Money Back, No Questions Asked Guarantee!
100%
Money Back Guarantee
You are fully protected by our 30 day money back guarantee. If you are not satisfied with your purchase, for any reason at all, simply contact us within 30 days of purchase and our helpful support staff will promptly issue a refund.
Conducive Consulting, Inc.
3445 Executive Center Dr Ste 216
Austin, TX 78759
512-551-0660
Copyright Conducive Consulting Inc 2019. For our Privacy Policy, please email info@conduciveconsulting.com