Can you tell us a bit about the project? Apache DolphinScheduler is a distributed and extensible open-source workflow orchestration platform with powerful DAG visual interfaces, which is agile in creating a high-performance workflow with low code. DolphinScheduler was donated to the Apache Software Foundation (ASF) in 2019, becoming an ASF Incubator…
This blog post demonstrates that native AMQP 1.0 in RabbitMQ 4.0 provides significant performance and scalability improvements compared to AMQP 1.0 in RabbitMQ 3.13. Additionally, this blog post suggests that AMQP 1.0 can perform slightly better than AMQP 0.9.1 in RabbitMQ 4.0. SetupThe following setup applies to all benchmarks in…
We often talk about how good, fast and reliable access to data is paramount if you want to have an upper hand in your AI/ML game. While different replication strategies allow you to access your data to be distributed across multiple nodes or sites, you also need to think about…
Our client, a global financial institution headquarterd in Japan, recently completed an ambitious Hadoop replacement project with MinIO and Dremio. You can see them present it in this talk from Subsurface but we thought we would write it up as well. Like most banks, the firm had built out a large…
One of the most exciting developments in data is the rise of lakehouse functionality across all major database vendors. Snowflake and SQL Server have long adopted this and now PostgreSQL is embracing this paradigm shift with pg_lakehouse, making it easier than ever to leverage modern datalakes for analytics, AI and…
Team RabbitMQ has two updates related to our Debian and RPM repositories: On August 18, 2024, Team RabbitMQ's PackageCloud account will be discontinued Cloudsmith mirror repositories now use *.rabbitmq.com domains, please update your repository definition files Mirrors Now Use *.rabbitmq.com Domains The docs were updated to use *.rabbitmq.com for mirror…
Amidst the excitement of AI and other new technologies, there's one component that quietly yet crucially holds everything together - literally as well as figuratively. That is modern object storage. It may not be glamorous, it is certainly not flashy, but it is the backbone of the modern datalake, making…
The regulatory landscape is evolving rapidly, and the upcoming Digital Operational Resilience Act (DORA) in Europe is a testament to this dynamic change. We have multiple European banking customers and each one is approaching the problem from a slightly different angle with one exception - almost all of them are…
OverviewMinIO Enterprise Object Store and Splunk have a symbiotic relationship when it comes to enterprise data. Splunk uses MinIO in its Digital Stream Processor. MinIO is a Splunk SmartStore endpoint.MinIO Enterprise Object Store is a high performance, Amazon S3 compatible, distributed object storage system. By following the methods and design…
Modern data processing environments move terabytes of data between the compute and storage nodes on each run. One of the fundamental requirements in a load balancer is to distribute the traffic without compromising on performance. The introduction of a load balancer layer between the storage and compute nodes as a…