Last updated on January 19, 2017
Recently I got opportunity to work and explore the event hubs. I managed to write about things that I have learnt about event hubs. This blog is going to be part of multi part series and the topics are going to be as follows:
- Introduction to Event Hubs
- Creating Event Hubs
- Message protocols in Event Hubs
- Sending Messages to Event Hubs
- Receiving Messages from Event Hubs
- Suggested Cloud Patterns when using Event Hubs
- Things to remember when using event Hubs
Event hubs (released for public preview in July 2014 and general availability during December 2014) are a hyper scale data ingress service. It is playing huge role in IoT platform where it is necessary to connect and collect data from millions of devices. The collected data can be used in data analytics, Machine learning and many number of ways that is useful to the business.
Some Key features of the event hubs are as follows:
- Event Hub guarantees high Throughput, high reliability and low latency. Event hubs don’t guarantee message sequence and should not be treated like a queue.
- Each message that is published into event hubs is called as Event. Each Event comprise of Event Body and a property bag that contains list of custom defined properties. There are some system properties such as offset that will also be part of the property bag.
- The Events in the event hubs cannot be deleted. The Events are retained in the hub based on predefined retention time interval and are deleted once the time elapses.
- The events can be batched together or published as a single event.
- The maximum size of a single dispatch can be up to 256 KB
Terminologies associated with the Event Hubs:
Namespace: A Namespace is a common identity for components under same service bus entity. The components can be queues, topics, Relays or Event Hubs. The Namespace URL with Entity Type can uniquely identify a particular entity.
Consumer Group: Consumer Group enables multiple consumers to process messages at the desired rate. Each consumer can maintain an offset by which they can determine from where they have to process the messages. The number of consumer groups vary from 1 (Default consumer) to 20 (in case of Standard tier)
Partitions: Event Hubs uses partitioned consumer pattern. Number of partitions for the event hub can be up to 32 and can be decided during the creation of the event hub. When a new message is arrived, it is placed at the end of one of the partition. The consumer of the event hub needs to read from all the partitions
Shared Access Signature (SAS): A Shared access signature is a way to provide authenticated access to the resource. SAS can be at Service bus namespace level or Event Hub level. The privileges can be read, send or manage
Event Checkpoint: A checkpoint is cursor that each consumer has to maintain in order to keep track of the position of the events that are read.
Partition Key: A partition key can be used by the event publishers organize the events so that consumers can use it in an organized manner.
In the next part of this series, we shall discuss about the creation of the Event Hubs.