Previously, I wrote about How I Processed 666K Pages Of Flattened PDFs into a Full Text Search Engine called the Apario writer. Upon on the conclusion of the last segment, I was able to optimize the compilation time of the original collection of data by rewriting the sidekiq Ruby pipeline script into a dedicated Go Application. Regardless of what compiling the PDF assets would look like, I still needed to serve those assets - and that's where the writer did little to nothing to actually address the OPEX of the project from 2020. Given the size of the data set, the 666K pages ended up compiling into a directory of ~1.13TB in size. This was held in storage that was distributed across several high volume storage dedicated servers on OVH behind MinIO. This provided an S3 compatible API directly.
What I Know About OPEX
OPEX or Operational Expense is how you describe a spending of money that is used explicitly for the operations of the business versus a capital expense. Hardware was considered a CAPEX or Capital Expense. So when Bit Fry Game Studios needed their DevOps pipeline upgraded for the 9 hour game builds into a 30 minute private enterprise cloud build, it required a CAPEX investment of $69K plus trust in me in order to achieve a -$15K/month OPEX savings. Annualized over a hardware lifecycle, over $472K can be recovered from OPEX by making a small CAPEX expense up front. One of the first projects that I ever worked on was in PHP and MySQL on Ubuntu 8.04. It was to balance the budget of a department that had ACME Bucks so to speak. It required me to write a finance module, fully tested, that managed Blue, Green and Black dollars. Blue dollars were for OPEX. Green dollars were for CAPEX. Black dollars were for external vendors where money left the company (versus moving between departments). Black depreciated instantly - meaning 100% of it was paid immediately. Blue dollars were borrowed over a 12 month pay-back period. Green dollars were borrowed over a 36 month pay-back period. The department was given $N per month of Blue, Green, and Black dollars, and there were n-Commitments existing on the books that were depreciating individually from past purchases that summed into the total monthly budget. I wrote a utility that would express this information plainly to multiple Director of Engineer colleagues that I collaborated with closely through a dashboard interface to see monies coming in and going out for the department. That was attached to greater projects and used elsewhere - but the lesson that stayed with me since those early days at Cisco was the use of budgeting between OPEX and CAPEX. Cisco's size and volume of diverse departments made Blue and Green dollars possible, but in other businesses that are smaller, that's OPEX and CAPEX. I've built tooling around reporting reliable OPEX & CAPEX information for Engineering Directors.
