Data Innovation with L2Bedford

Teams faced with the challenge of building systems to support the advent of the Big Data revolution face a bewildering array of difficult choices if they are to successfully exploit the promise and potential of ubiquitous cloud resources. The strategies and techniques required to confidently manage data assets in the cloud are non-trivial and not obvious; adopting them requires mastery of the underlying distributed architecture and technologies. Lacking in-house expertise, the playground of cloud offerings and shiny open source tools, oftentimes results in risky and fruitless PoCs, or unwieldy, unreliable Frankenstein solutions that fail to deliver the promised cost-savings and flexibility. At the same time, much of the playbook for data management no longer applies, but a professional appreciation for its legacy is necessary to recognize when to break the rules and when to establish new ones.
Read More

The Problem with Big Data.

The advent of cloud computing unleashed the promise and potential of the Big Data revolution. The availability of on-demand, elastic computing and cheap, abundant storage has elevated the practice of data integration and analytics to a science. Data science and the big data that fuels it are the most pervasive and impactful technical trends of the early 21st century. No longer is the collection of operational data and its formulation into impactful business insights constrained by costly and measly data center storage limits or governed by small windows of overnight batch processing. The ability to manage Big Data has enabled massive innovation, giving rise to data-driven companies and products hitherto unimaginable. Yet what was until recently considered bleeding edge is now merely table stakes for any serious contender in the business of data. Data-driven organizations demand a dependable data infrastructure. Engineering teams building systems to support this data imperative face a bewildering array of difficult choices if they are to successfully exploit ubiquitous cloud resources. Lacking in-house expertise or a vision for data, the playground of cloud offerings and open-source tools oftentimes results in risky and fruitless PoCs; or unwieldy, unreliable Frankenstein solutions that fail to deliver the expected cost-savings and flexibility. The techniques required to confidently manage data assets and elastically provisioned resources that both scale-out and scale-in are non-trivial and not obvious; adopting them requires mastery of the underlying distributed architecture and technologies. Naive implementations having tightly bound storage and compute, squander the cost-savings opportunity of the pay-as-you-go model. Moreover, much of the established playbook for data management no longer applies, but a professional appreciation for its legacy is necessary to recognize when to break the rules and when to establish new ones.
Read More