In computer science, streaming algorithms process input data streams as a sequence of items, typically making just one pass (or a few passes) through the data. These algorithms are designed to operate with limited memory, generally logarithmic in the size of the stream and/or in the maximum value in the stream, and may also have limited processing time per item. As a result of these constraints, streaming algorithms often produce approximate answers based on a summary or "sketch" of the data stream.
History Though streaming algorithms had already been studied by Munro and Paterson as early as 1978, as well as Philippe Flajolet and G. Nigel Martin in 1982/83, the field of streaming algorithms was first formalized and popularized in a 1996 paper by Noga Alon, Yossi Matias, and Mario Szegedy. For this paper, the authors later won the Gödel Prize in 2005 "for their foundational contribution to streaming algorithms." There has since been a large body of work centered around data streaming algorithms that spans a diverse spectrum of computer science fields such as theory, databases, networking, and natural language processing. Semi-streaming algorithms were introduced in 2005 as a relaxation of streaming algorithms for graphs, in which the space allowed is linear in the number of vertices n, but only logarithmic in the number of edges m. This relaxation is still meaningful for dense graphs, and can solve interesting problems (such as connectivity) that are insoluble in o ( n ) {\displaystyle o(n)} space.
Models
Data stream model In the data stream model, some or all of the input is represented as a finite sequence of integers (from some finite domain) which is generally not available for random access, but instead arrives one at a time in a "stream". If the stream has length n and the domain has size m, algorithms are generally constrained to use space that is logarithmic in m and n. They can generally make only some small constant number of passes over the stream, sometimes just one.
Turnstile and cash register models Much of the streaming literature is concerned with computing statistics on frequency distributions that are too large to be stored. For this class of problems, there is a vector a = ( a 1 , … , a n ) {\displaystyle \mathbf {a} =(a_{1},\dots ,a_{n})}
(initialized to the zero vector 0 {\displaystyle \mathbf {0} } ) that has updates presented to it in a stream. The goal of these algorithms is to compute functions of a {\displaystyle \mathbf {a} } using considerably less space than it would take to represent a {\displaystyle \mathbf {a} } precisely. There are two common models for updating such streams, called the "cash register" and "turnstile" models. In the cash register model, each update is of the form ⟨ i , c ⟩ {\displaystyle \langle i,c\rangle } , so that a i {\displaystyle a_{i}} is incremented by some positive integer c {\displaystyle c} . A notable special case is when c = 1 {\displaystyle c=1}
(only unit insertions are permitted). In the turnstile model, each update is of the form ⟨ i , c ⟩ {\displaystyle \langle i,c\rangle } , so that a i {\displaystyle a_{i}} is incremented by some (possibly negative) integer c {\displaystyle c} . In the "strict turnstile" model, no
a i {\displaystyle a_{i}} at any time may be less than zero.
Sliding window model Several papers also consider the "sliding window" model. In this model, the function of interest is computing over a fixed-size window in the stream. As the stream progresses, items from the end of the window are removed from consideration while new items from the stream take their place. Besides the above frequency-based problems, some other types of problems have also been studied. Many graph problems are solved in the setting where the adjacency matrix or the adjacency list of the graph is streamed in some unknown order. There are also some problems that are very dependent on the order of the stream (i.e., asymmetric functions), such as counting the number of inversions in a stream and finding the longest increasing subsequence.
Evaluation
The performance of an algorithm that operates on data streams is measured by three basic factors:
The number of passes the algorithm must make over the stream. The available memory. The running time of the algorithm. These algorithms have many similarities with online algorithms since they both require decisions to be made before all data are available, but they are not identical. Data stream algorithms only have limited memory available but they may be able to defer action until a group of points arrive, while online algorithms are required to take action as soon as each point arrives. If the algorithm is an approximation algorithm then the accuracy of the answer is another key factor. The accuracy is often stated as an ( ϵ , δ ) {\displaystyle (\epsilon ,\delta )} approximation meaning that the algorithm achieves an error of less than ϵ {\displaystyle \epsilon } with probability 1 − δ {\displaystyle 1-\delta } .
… excerpt ends here. Continue reading the full article.
