...

Managed Service for Apache Kafka

Deploy an Apache Kafka cluster in just a few clicks with our fully managed service. Focus on building your applications while we take care of the right configurations, monitoring, backups and updates.

Calculate cost

Solve Your Challenges with the Managed Service for Apache Kafka

Distributed Event Processing

Process numerous events in real time, ensuring horizontal scalability and fault tolerance.

Centralized Logging

Aggregate logs from various sources for analysis, debugging, and compliance.

Reactive Architecture

Build an asynchronous system that can handle data streams with low latency.

Logistics and Telematics Solutions

Manage data flow from logistics and telematics sources, enabling location, transport, and route tracking and analytics.

Microservices Architecture

Create a flexible, scalable microservices architecture, exchanging messages through Apache Kafka.

Why Apache Kafka

A reliable, scalable platform for real‑time data‑stream processing:

  • High throughput and low latency: Apache Kafka can handle millions of messages per second with quick response, ensuring instant data delivery.
  • Fault tolerance and data redundancy: Apache Kafka delivers high data availability and background replication, safeguarding data even during system failures.
  • Horizontal scaling with virtually limitless growth: Apache Kafka easily handles processing and storing massive data volumes without performance loss.
  • Integration with diverse data sources: Apache Kafka lets you combine data from various sources for analysis and processing.
  • Flexible and extensible framework: Apache Kafka supports multiple data types and formats and enables custom extension development.
  • Ease of use with SQL: Apache Kafka is accessible and convenient for data analysts and developers thanks to SQL support.

What Tasks We Handle for You

Our platform provides convenient, flexible administration of managed databases alongside reliability and high performance. We take on the complexity of installing and configuring the databases, freeing up your time to focus on growing your business. Plus, our team of experts is always ready to help with any questions.

  • Virtual Machine Deployment
  • OS and Software Installation
  • Data Storage and Hardware Security
  • Network Configuration
  • Broker Updates
  • Monitoring Tools

Other databases

DBaaS (Database as a Service) is a cloud-based approach to data storage and management in which you no longer need to install or maintain the database yourself. Instead, you receive a ready-to-use, fully optimised cloud solution that delivers high availability, effortless scalability, and comprehensive database management.

  • MongoDB

    MongoDB

    A high-performance NoSQL database ideal for flexible, efficient management of large volumes of data.

    Discover
  • Elasticsearch

    Elasticsearch

    A powerful search and analytics engine for indexing large volumes of data and handling real-time queries.

    Discover
  • PostgreSQL

    PostgreSQL

    A massively parallel processing (MPP) platform for large-scale data warehousing and analytics.

    Discover
  • Greenplum

    Greenplum

    A massively parallel processing (MPP) platform for large-scale data warehousing and analytics.

    Discover
  • MySQL

    MySQL

    The world’s most popular open-source database, renowned for its reliability, simplicity and speed.

    Discover
  • Redis

    Redis

    An in-memory data-structure store that serves as a database, cache and message broker for lightning-fast data access.

    Discover
  • ClickHouse

    ClickHouse

    A high-performance columnar DBMS optimised for online analytical processing (OLAP).

    Discover
  • OpenSearch

    OpenSearch

    An open search-and-analytics suite used for real-time log monitoring and analysis.

    Discover

Have a complex setup or additional pricing questions? Contact our sales team to get the information you need.

Want more capabilities?
Full‑fledged OpenStack infrastructure is ready

For individuals and companies of any size

Your needs define our cloud, which adapts and scales to your workloads. Start with a small virtual machine and effortlessly grow into high‑load infrastructures with clusters, load balancers, and other tools.

For efficient operations

Deploy virtual machines, configure networks and complex network architectures, manage data storage with great flexibility, and achieve high availability using load balancers and clusters. All of this — in an intuitive interface. With it, you can focus on what matters most — your business and ideas.

For Bare Metal with maximum power

If you need maximum performance and control, we offer you the option of deploying OpenStack on Bare Metal. It is the key solution for high‑performance computing, big data processing, and situations where direct access to hardware resources is required.

Build your cloud

Sign up now and be up and running in just a few minutes.

Begin

Other products you might be interested in

Kubernetes (K8s)

An orchestration system that automates the deployment, scaling and management of containerised applications.

Try

FAQ

What is Apache Kafka and for what tasks is it suitable?

Apache Kafka is a distributed, horizontally scalable software message broker designed to organize the collection and storage of streaming data, its processing in real time with high throughput and minimal latency. Streaming data refers to data that is continuously generated by a large number of sources that typically send data records in small volumes. Examples of streaming data include log files generated by customer applications, online shopping information, user actions in games, social network activity, marketplace data, geospatial services, and telemetry data. Apache Kafka was developed to efficiently handle, process, and respond to such data streams.

Kafka is an open-source distributed system based on client-server architecture. Data is exchanged within a Kafka cluster using its own binary data transfer protocol designed to reduce the overhead of data transfer.

An Apache Kafka cluster consists of servers and clients. Some servers form the storage layer and are known as brokers, while others run the Kafka Connect mechanism and continuously import and export data as event streams. Clients host distributed applications and microservices, which work in parallel to receive, process, and generate event streams.

The Kafka cluster is fault-tolerant; if a server fails, other servers will take over to ensure uninterrupted operation without data loss.

Apache Kafka can be used wherever event stream acquisition and processing is required. Kafka is used in a wide variety of industries, but is most commonly used for the following tasks:

  • Real-time processing of financial transactions and payment information, such as at stock exchanges and by banks and insurance companies. For example, Kafka is used by organizations such as ING Bank.
  • Tracking and monitoring user, transportation, and cargo locations in real time using geopositioning systems. For instance, Kafka supports messaging and data integration for Foursquare’s Big Data infrastructure.
  • Continuously collecting and analyzing data from sensors, telemetry, devices, controllers, or other equipment, making it ideal for implementing IoT/IIoT systems.
  • Building data pipelines for analytics systems to extract useful information from raw data using machine learning algorithms. Kafka is employed by companies like IBM and DataSift for real-time data collection from event streams.
  • Collecting and immediately responding to customer interaction events, such as in retail, hospitality, mobile apps, and online gaming. Kafka provides the backbone for organizing any event-driven data platforms, architectures, and microservices.

What versions of Apache Kafka does Managed Service for Apache Kafka support?

Our service currently supports Apache Kafka versions 2.8.1 and 3.1.0.

Can a user upgrade the version of an Apache Kafka cluster?

The user can upgrade the cluster to any supported version by selecting the desired version in the cluster properties. However, the cluster can only be upgraded upwards and once the upgrade is done, the cluster cannot be reverted back to the previous version.

Before upgrading the version, ensure that the client software will allow you to use the features of the new version of Apache Kafka cluster. As part of the cluster version upgrade, only the server software will be upgraded, not the client software.

It is recommended to upgrade the cluster to the next version relative to the current version, for example, to upgrade Apache Kafka from version 2.8 to 3.1 you would upgrade in the following sequence: upgrade the application from version 2.8 to version 3.0 and then upgrade from version 3.0 to 3.1.

I got an error message: "disk size must be at least ... according to topics partitions number and replication factor, but size is ...", what to do?

The error is caused by topic log segments taking up more space than is available in the broker storage. To solve this problem, you can increase the size of disk storage or reduce the segment size for topics or the whole cluster.