Open Source PostgreSQL On Steroids: Swarm64 Database Acceleration Software For Performance Improvement And Analytics

PostgreSQL is a big deal. The most common SQL open source database that you have never heard of, as ZDNet's own Tony Baer called it. Besides being the framework on which a number of commercial offerings were built, PostgreSQL has a user base of its own. According to DB Engines, PostgreSQL is the 4th most popular database in the world.

Swarm64, on the other hand, is a small vendor. So small, actually, that we have shared the stage with CEO Thomas Richter in a local Berlin Meetup a few years back. Back then, Richter was not CEO, and Swarm64 was even smaller. But its value proposition still sounded attractive: boost PostgreSQL's performance for free. Swarm64 is an acceleration layer for PostgreSQL.

There's no such thing as a free lunch of course, so the "for free" part is a figure of speech. Swarm64 is a commercial vendor. Until recently, however, the real gotcha was hardware: Swarm64 Database Acceleration (DA) required a specialized chip called FPGA to be able to do its PostgreSQL magic. With Swarm64 DA 4.0 released today, that's no longer the case.

Accelerating PostgreSQL

First, a little bit of history. Why choose to accelerate PostgreSQL? Certainly, an open source database makes sense for many reasons, and PostgreSQL is popular. So to make it more concrete: why not MySQL, PostgreSQL's rival, which is even more popular?

When Swarm64 DA was initially developed, the question was considered, said Richter. PostgreSQL was finally chosen, "because it's a wonderfully extensible database, which helped us bring Swarm64 DA to market sooner -- as an extension to the database rather than a fork".

Having been acquainted with earlier versions of Swarm64 DA, the difference in this one stood out: no FPGA necessary. Richter ascertained that Swarm64 DA 4.0 is a software-only solution, requiring no hardware upgrades whatsoever. So how does it work then?

A primary feature of Swarm64 DA is its ability to add more parallel processing to PostgreSQL queries, and that can be done with or without the FPGA, said Richter. In lab tests, he went on to add, CPU-only acceleration on a 1 terabyte TPC-H database is 10x; adding an FPGA nearly doubles that. The FPGA benefits, relative to CPU only, increase as the database size or number of concurrent users rises.

Before going any further, a word on benchmarks and use cases. TPC-H mentioned by Richter is a benchmark designed to evaluate the performance of systems used for analytics related queries. While Swarm64's is about performance improvement, the release of DA 4.0 marks a focus on analytics.

The message seems to be that PostgreSQL plus Swarm64 DA 4.0 can be used as a data warehouse solution. Richter also noted that FPGAs enable people to use a single database to support data collection and reporting at the same time: "CPUs put data into the DBMS, while FPGA handles the reporting and analytics. We commonly see this in IoT, GIS use cases, performing real-time analytics".

Richter went on to add that Swarm64 DA accelerates PostgreSQL by employing several techniques. Columnar indexing is one, but so is adding much greater parallel processing to query execution, and data compression, which helps reduce time-consuming I/O. This release also includes accelerated SQL joins and text search:

"These are the major ingredients in most data warehouse or analytic databases, and we add them to PostgreSQL to help companies implement systems of insight much more easily and less expensively".

It never hurts to help people reduce costs

PostgreSQL is relational database designed for operational, transactional workloads. While convenience is part of the reason why people may want to stick with their PostgreSQL for analytics workloads, cost is another one. 

The value proposition for Swarm64 seems to be "keep your open source, free PostgreSQL, make it run faster". Swarm64 DA's price point is $33 / vCore / month. This may sound compelling, but one should also wonder how many of Swarm64's clients really only use PostgreSQL without any additional support or other services. Including the cost of in-house resources and expertise, the real TCO is hidden, and adds up.

Richter noted that the real benefit of Swarm64 DA is enabling people to use free, open source PostgreSQL in more projects, especially in analytics and data warehousing, where its adoption has not been as strong historically:

"Using PostgreSQL for these projects helps people save a ton of money on commercial DBMS license costs; the annual DBMS maintenance fee paid to a vendor like Oracle can reach 6-figures. Using PostgreSQL for free, eliminates that -- other costs like DB administrators and tools are expenses people bear whether they use commercial DBs or free ones.

The cost of using Swarm64 DA-accelerated PostgreSQL pales in comparison to commercial on-premises or cloud-based data warehouse platforms--about a third the total cost of using Amazon Redshift over 3 years, and nearly 10x less than most legacy data warehouse platforms".

That math is something to consider for every potential user with more use cases in the area of data warehouse modernization, new machine learning and data science systems, and real-time analytics (IoT, Time Series, GIS), which Richter said are the ones they encounter the most.

As Richter noted, it never hurts to help people reduce costs, and cost reduction has always been a big reason for the success of open source database adoption. The interest in saving money in new ways with Swarm64 DA-accelerated PostgreSQL is very strong in the current climate, just as it was before the pandemic, he went on to add.

Going forward

This release seems to have the potential to make the combination of PostgreSQL plus Swarm64 DA 4.0 a contender for analytics workloads. Richter mentioned that Swarm64 is focused on PostgreSQL for the foreseeable future, but adding support for MariaDB and MySQL or other databases at some point is a possibility.

In addition, Swarm64 does not forget its roots, and keeps an eye on hardware. Swarm64 has a dual relationship with Intel and Xilinx: they are both partners and investors. Besides joint business development, Richter said, those partnerships provide Swarm64 with an inside track on new hardware innovations:

"There's a lot happening in hardware -- FPGAs of course, but also persistent RAM, computational storage, and edge computing. These are potential game-changers in analytic and IoT data management, and our hardware partnerships enable us to help customers gain competitive performance advantages".

DA 4.0 offers triple the speed of Swarm64 DA 3.0, but Richter thinks there still is room for improvement. Besides finding new ways to make PostgreSQL faster, the focus is on ease of use, to make it even easier for PostgreSQL users to benefit from Swarm64 DA.

Swarm64 has also developed a few open source projects that help PostgreSQL users accelerate, scale, and measure performance. Within the last year, patches for elastic scaling and data backup/restore were contributed, and the plan is to continue development on those.