https://www.ibm.com/topics/apache-spark https://www.confluent.io/ CEP = conceptual hierarchies and causal relationships Similar to stream processing, complex event processing (CEP) is an event-driven technology for aggregating, processing, and analyzing data streams in order to gain real-time insights from events as they occur. Benefits of CEP Today, companies are flooded with facts that they don’t know how to use. CEP separates the chaff from the grain by transforming low-level data into high-level business information that companies care about. In this way, CEP enables companies to take charge of external events as they happen in real-time. Complex event processing is a generalization of traditional stream processing. Traditional stream processing is concerned with finding low-level patterns in data, such as the number of mouse clicks within a fifteen-minute window. CEP promises much more. Using models of causality and conceptual hierarchies, CEP can make high-level inferences about complex events within the business domain. One of the distinguishing features of CEP is the use of conceptual hierarchies. In the real world, events happen at different levels of abstraction. At one level, we can talk about a customer’s intentions and feelings. At a lower level, we can talk about her GPS trail or the actions of her mouse. When we are dealing with two or more levels at once, we cannot expect to process a simple time-ordered stream of events. Instead, we must be able to consume “an event cloud". The difference between an event cloud and an event stream is that the event cloud contains data from many streams at different levels of abstraction. The CEP toolkit can identify complex patterns in such a multi-level system. Another distinguishing feature of CEP is the use of causal relationships. Suppose we are searching for a business pattern in which a certain combination of GPS movements and mouse clicks is expected to cause a business event such as a purchase or a cancelation. With CEP, we can choose to flag this combination as a complex event. However, we must not assume that the data will always arrive in the correct time sequence. When we perform the search for the complex event, it may happen that the user data arrives earlier than the GPS data. In this case, we must be able to remember the first part of the complex event while continuing to search for the remaining parts. The key benefit of CEP is that actions can be triggered by a combination of events happening at different times and in different contexts. Source: https://www.confluent.io/learn/complex-event-processing/ LLMs are really good at this: ``` What's happening? church bells ringing. the appearance of a man in a tuxedo with a woman in a flowing white gown. rice flying through the air. Respond in the format: <Event or phrase> ``` ``` <Wedding> ``` or ``` Complex Event Processing (CEP). Summarize all the headlines into one event. Respond in the correct format. MrBeast faces allegations over potential child labour in Feastables Chocolate production tribune.com.pk 1 month ago (8/24/2024 10:30am) English Pakistan MrBeast Has Now Accomplished Something No Other YouTuber Has Ever Done gamerant.com 2 months ago (7/10/2024 4:00pm) English United States ... YouTuber MrBeast Says He Will Jump In Race For US Presidency If This Happens ndtv.com 2 months ago (7/10/2024 2:45pm) English India Tell me a single event. Respond in the format: <Event or phrase> ```