Pub/Sub in your DB? Oracle Database TxEventQ

Pub/Sub in the Database? Yes — Here’s Why.

In this article, we’ll discuss how to use your database for event streaming with Oracle Database Transactional Event Queues (TxEventQ), a built-in, database-native message broker.

TxEventQ supports multiple consumers/producers, exactly-once messaging, partitioned queuing (key based and automatic), and strict ordering per message stream.

TxEventQ is included in every Oracle Database version from 21c onward, including Oracle Database Free.

Flowchart illustrating the timeline of Oracle Database features related to AQ (Advanced Queuing) from version 8.0 to 23ai, highlighting key developments and changes in naming and functionality.
A little history: Oracle Database added database messaging with Advanced Queuing in 1997, which later evolved into sharded event streams, what we now call Transactional Event Queues.

What are Transactional Event Queues?

TxEventQ is built into the database and backed by a specialized, sharded table architecture.

This means event streams in Oracle Database can take full advantage of advanced database features:

Illustration showing a consumer group on the left, a database icon in the center with arrows indicating data flow, and an Oracle Database version 23ai label on the right.

Client APIs work anywhere you have a database client – in common languages like Java, Python, Javascript, C, .Net, and PL/SQL.

Familiar with Apache Kafka? Use the Kafka Java Client for Oracle Transactional Event Queues, or produce/consume events over REST with Oracle REST Data Services (ORDS).

High-throughput installations of Transactional Event Queues have been benchmarked at up to 1 million messages per second on an 8-node Oracle RAC Database.

Why use Transactional Event Queues instead of another message broker?

There are many messaging systems on the market, both proprietary and open-source, with different goals, capabilities, and limitations:

TxEventQ shines when you want to leverage the existing capabilities of the database for robust, exactly-once messaging:

  • Simplified Infrastructure:
    • No separate Kafka cluster or heavy middleware required.
    • Runs where your Oracle Database runs, minimizing moving parts.
  • SQL-Native Compatibility:
    • Event streaming, CDC, and queries work with familiar SQL.
    • Developers don’t have to learn a new API surface to participate in event-driven systems.
  • Transactional Messaging:
    • Database changes and event publishing happen atomically with transactional integrity.
    • Guarantees exactly-once semantics, solving the dual-write problem.

Why have some users adopted TxEventQ?

Developers choose TxEventQ because it solves common pain points in building event-driven applications:

  1. Running your message broker in the database allows you to eliminate the need for separate, costly servers dedicated to event streaming. These servers require domain-specific knowledge to operate, maintain, and upgrade in production deployments.
  2. Combining messaging operations with inserts/updates of related data in a single transaction is incredibly simple and powerful: no headaches about data loss or corruption.
  3. The transactional integrity of TxEventQ is incredibly important to users working with data that requires exactly-once semantics: They need strict guarantees and transactional messaging.

Teams that need simplicity, reliability, and tight synchronization between data and events adopt TxEventQ because it reduces moving parts and keeps messaging as reliable as the database itself.

How to build apps using TxEventQ?

If you’re using Java, start with the Kafka Java Client for Oracle Transactional Event Queues — Familiar Kafka APIs make working with TxEventQ simple when creating topics, publishing events, and consuming events.

It’s also easy to convert existing Kafka Java apps to Transactional Event Queues, given that the API is identical.

Sample applications using TxEventQ

These code samples help you get started developing applications using TxEventQ using various paradigms (REST, SQL, clients) and programming languages.

Transactional Event Queue SDK and implementation references

Oracle maintains a number of other clients and reference information for using Transactional Event Queues. Most Oracle Database drivers (Java, Python, NodeJS, etc.) include support for Transactional Event Queues.

Questions? Let me know in the comments section, or reach out directly.

Responses

  1. João Avatar

    Hi Anders, is there any way to automatically produce messages to txeventq on table DML operations, or is it fully required to use triggers?

    1. Anders Swanson Avatar

      Hi João,

      Triggers are the easiest way to set this up – you can create the appropriate DML trigger in the database, and then use the dbms_aq.enqueue method in the trigger function to publish the message to your specific queue or topic.

      See Event-driven CDC, all within the database for a code example using TxEventQ and triggers.

  2. […] curation and selection can take time. For agents using Oracle AI Database, OKafka and Transactional Event Queues (TxEventQ) enable asynchronous processing of memory candidates. A worker can extract memory candidates, apply […]

  3. […] APIs to create topics, produce events, and consume events. Under the hood, it’s backed by Oracle AI Database Transactional Event Queues (TxEventQ), a high-throughput messaging system built into the […]

Leave a Reply