We started Feldera to build a fast data computational engine for continuous analytics. This means developers can use their existing SQL as standing queries for comprehensive non-stop results directly from their data in motion.
We are also pleased to announce that Tony Liu at Costanoa has graciously invested a $6M seed funding round to accelerate our work.
🧬 Our story
In 2018, when Leonid and Mihai first started building an incremental data processing library, Differential Datalog at VMware Research, there was no robust library or system for continuous analytics on streaming data.
As Differential Datalog starting gaining adoption, it attracted additional committers and management noticed. Soon Gerd, Ben, Lalith and Chase jumped onto the open-source effort to help accelerate features.
Around 2020, with Mohsin's help, the library saw deployment across thousands of customers for continuous analytics in VMware Skyline's cloud service. Observing how developers implemented their queries and rules, we quickly realized a better developer user experience was needed. And SQL was familiar and well understood.
That got Mihai and Leonid back to the drawing board. Their hard work resulted in the invention of a streaming algebra, Database Stream Processor (DBSP) in 2022. This created a solid mathematical and academic foundation for continuous analytics over data in-motion.
Feeling empowered, we doubled down our efforts over the next year. We soon built a compiler for standing SQL queries using Apache Calcite, a blazingly fast data processing engine built in Rust to execute the queries, a containerized runtime, hooks for storage, just-in-time compiler, a distributed model design, multi-tenancy runtime isolation mechanisms, and much more.
Seeing it in production in more use cases we realized our vision of building something bigger required a much larger effort.
⭐ A Better Way of Doing Analytics
Continuous analytics has been around for a long time, with many claiming to do it. With Feldera, we coalesced the architecture and features into our Continuous Analytics Platform. What makes Feldera's engine unique is its ability to evaluate arbitrary SQL programs incrementally, making it more expressive and performant than existing alternatives like streaming engines.
With the platform, developers configuring data pipelines are not exposed to the complexities of querying changing data, an otherwise notoriously hard problem. Instead, they can express their analytics and transformations as standing queries and have the platform evaluate these queries incrementally, correctly and efficiently.
Over the past few months, we've been excited to find that developers care deeply about those capabilities. And they want even more!
Every time we talked to developers, we were asked about features ranging from reusing existing OLTP or OLAP SQL data models as-is in Feldera, to unifying analysis different formats and sources of data, to making existing data transport faster. Yet we were taken aback when we realized that modern data stack toolchains are merely focused on moving lots of data in or out of datastores. They don’t do anything that facilitates getting value out of the data.
Armed with these insights, we defined out product's north star:
- ⚙️ standard SQL as the initial query language, and
- 🪄 a delightful interactive UI for building continuous analytics on data in motion, and
- 📦 a containerized runtime implementation for deployment
In addition, the platform retains its focus on developer friendly roots by having a git-style change control process. And our core DNA of extreme performance means Feldera Analytics Platform is able to do even more analytics with each new release.
For production deployments, we are building a paid service that offers a Bring Your Own Cloud (BYOC) form factor. The BYOC offering allows Feldera to be run in a developer owned Kubernetes environment, as the developer or architect sees fit.
We are firing on all cylinders to deliver an extraordinary solution to developers. All while keeping Feldera Analytics Platform free and open-source under the MIT License.
📣 Developer Preview availability
As an engineer myself, I understand that adopting a new product or technology is a complex decision. To that end, today we are announcing the Developer Preview version available for everyone to download ⬇️ and try. We view this milestone as a source of motivation and pride for what we have been able to achieve so quickly.
📬 Remain in touch
Beyond the excitement of our product and company, I am personally amazed by the level of expertise, trust and shared purpose on our journey. I feel blessed to work on something that resonates so strongly with our users - so thank you co-founders, employees, advisors, users and investors for your confidence and support.
As always you can find the core team on Feldera GitHub. We will also be at VLDB 2023 conference presenting DBSP. If you are going, please stop by our table.
Finally, if you think anyone can benefit from Feldera, have ideas on what you
would like to see,
contribute towards
features and issues, or just want to chat, I would love to hear from you. I am
always available at mohsin.beg@feldera.com.
Thank you.