Fundamentals of stream processing: application design, systems, and analytics
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Buch |
| Sprache: | English |
| Veröffentlicht: |
Cambridge [u.a.]
Cambridge Univ. Press
2014
|
| Schlagworte: | |
| Online-Zugang: | Inhaltsverzeichnis |
| Beschreibung: | XXVII, 529 S. Ill., graph. Darst. |
| ISBN: | 9781107015548 |
Internformat
MARC
| LEADER | 00000nam a22000002c 4500 | ||
|---|---|---|---|
| 001 | BV041716948 | ||
| 003 | DE-604 | ||
| 005 | 20160329 | ||
| 007 | t | ||
| 008 | 140304s2014 ad|| |||| 00||| eng d | ||
| 016 | 7 | |a 016433827 |2 DE-101 | |
| 020 | |a 9781107015548 |9 978-1-107-01554-8 | ||
| 035 | |a (OCoLC)871294049 | ||
| 035 | |a (DE-599)HBZHT017848400 | ||
| 040 | |a DE-604 |b ger | ||
| 041 | 0 | |a eng | |
| 049 | |a DE-473 |a DE-83 | ||
| 082 | 0 | |a 004.2/2 |2 23 | |
| 084 | |a ST 274 |0 (DE-625)143641: |2 rvk | ||
| 100 | 1 | |a Andrade, Henrique C. M. |e Verfasser |0 (DE-588)1048709507 |4 aut | |
| 245 | 1 | 0 | |a Fundamentals of stream processing |b application design, systems, and analytics |c Henrique C. M. Andrade, Buǧra Gedik, Deepak S. Turaga |
| 264 | 1 | |a Cambridge [u.a.] |b Cambridge Univ. Press |c 2014 | |
| 300 | |a XXVII, 529 S. |b Ill., graph. Darst. | ||
| 336 | |b txt |2 rdacontent | ||
| 337 | |b n |2 rdamedia | ||
| 338 | |b nc |2 rdacarrier | ||
| 650 | 0 | 7 | |a Hochleistungsrechnen |0 (DE-588)4532701-4 |2 gnd |9 rswk-swf |
| 650 | 0 | 7 | |a Streaming |g Kommunikationstechnik |0 (DE-588)4614855-3 |2 gnd |9 rswk-swf |
| 653 | |a Data flow computing. | ||
| 653 | |a SIMD (Computer architecture) | ||
| 689 | 0 | 0 | |a Hochleistungsrechnen |0 (DE-588)4532701-4 |D s |
| 689 | 0 | 1 | |a Streaming |g Kommunikationstechnik |0 (DE-588)4614855-3 |D s |
| 689 | 0 | |5 DE-604 | |
| 700 | 1 | |a Gedik, Buǧra |e Verfasser |0 (DE-588)1048660338 |4 aut | |
| 700 | 1 | |a Turaga, Deepak S. |e Verfasser |0 (DE-588)1048660508 |4 aut | |
| 856 | 4 | 2 | |m Digitalisierung UB Bamberg - ADAM Catalogue Enrichment |q application/pdf |u http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=027164019&sequence=000002&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA |3 Inhaltsverzeichnis |
| 999 | |a oai:aleph.bib-bvb.de:BVB01-027164019 | ||
Datensatz im Suchindex
| _version_ | 1804151992583454720 |
|---|---|
| adam_text | Contents
Preface
page
xiii
Foreword
xix
Acknowledgements
xxi
List of acronyms
xxii
Part I Fundamentals
ι
1
What brought us here?
3
I
.
I Overview
3
1.2
Towards continuous data processing: the requirements
3
1.3
Stream processing foundations
6
1.3.1
Data management technologies
8
1.3.2
Parallel and distributed systems
13
1.3.3
Signal processing, statistics, and data mining
16
1.3.4
Optimization theory
18
1.4
Stream processing
—
tying it all together
22
References
24
2
Introduction to stream processing
33
2.1
Overview
33
2.2
Stream Processing Applications
33
2.2.1
Network monitoring for cybersecurity
34
2.2.2
Transportation grid monitoring and optimization
36
2.2.3
Healthcare and patient monitoring
38
2.2.4
Discussion
40
2.3
Information flow processing technologies
40
2.3.1
Active databases
41
2.3.2
Continuous queries
42
2.3.3
Publish-subscribe systems
42
2.3.4
Complex event processing systems
43
2.3.5
ETL and
SCADA
systems
44
2.4
Stream Processing Systems
45
2.4.1
Data
45
2.4.2
Processing
49
2.4.3
System architecture
53
viii Contents
:»
.....
;;Л?
J -í
*:.:..
і:
.....
2.4.4
Implementations
56
2.4.5
Discussion
66
2.5
Concluding remarks
68
2.6
Exercises
69
Re
ferences
70
Part II Application development
75
3
Application development
-
the basics
77
3.1
Overview
77
3.2
Characteristics of SPAs
77
3.3
Stream processing languages
80
3.3.1
Features of stream processing languages
80
3.3.2
Approaches to stream processing language design
83
3.4
Introduction to SPL
86
3.4.1
Language origins
86
3.4.2
A Hello World application in SPL
87
3.5
Common stream processing operators
92
3.5.1
Stream relational operators
92
3.5.2
Utility operators
96
3.5.3
Edge adapter operators
97
3.6
Concluding remarks
101
3.7
Programming exercises
101
References
103
4
Application development
-
data flow programming
106
4.1
Overview
106
4.2
Flow composition
106
4.2.1
Static composition
108
4.2.2
Dynamic composition
112
4.2.3
Nested composition
122
4.3
Flow manipulation
128
4.3.1
Operator state
128
4.3.2
Selectivity and
arity
131
4.3.3
Using parameters
132
4.3.4
Output assignments and output functions
134
4.3.5
Punctuations
136
4.3.6
Windowing
138
4.4
Concluding remarks
144
4.5
Programming exercises
144
References
147
5
Large-scale development- modularity, extensibility, and distribution
148
5.1
Overview
148
Contents ix
!
¿ЇЙ
lÍÍ.ď,^
-ц-
,
rf..
.
,
.
.ν
5.2
Modularity
and extensibility
148
5.2.1
Types
149
5.2.2
Functions
151
5.2.3
Primitive operators
153
5.2.4
Composite and custom operators
161
5.3
Distributed programming
164
5.3.1
Logical versus physical How graphs
164
5.3.2
Placement
166
5.3.3
Transport
170
5.4
Concluding remarks
172
5.5
Programming exercises
173
References
176
6
Visualization and debugging
178
6.1
Overview
178
6.2
Visualization
178
6.2.1
Topology visualization
179
6.2.2
Metrics visualization
184
6.2.3
Status visualization
185
6.2.4
Data visualization
186
6.3
Debugging
188
6.3.1
Semantic debugging
189
6.3.2
User-defined operator debugging
194
6.3.3
Deployment debugging
194
6.3.4
Performance debugging
195
6.4
Concluding remarks
199
References
200
Part III System architecture
201
7
Architecture of a stream processing system
203
7.1
Overview
203
7.2
Architectural building blocks
203
7.2.1
Computational environment
204
7.2.2
Entities
204
7.2.3
Services
206
7.3
Architecture overview
207
7.3.1
Job management
207
7.3.2
Resource management
208
7.3.3
Scheduling
209
7.3.4
Monitoring
210
7.3.5
Data transport
211
7.3.6
Fault tolerance
212
7.3.7
Logging and error reporting
213
Contents
7.3.8
Security and access
control
213
7.3.9
Debugging
214
7.3.10
Visualization
214
7.4
Interaction with the system architecture
215
7.5
Concluding remarks
215
References
215
8
InfoSphere Streams architecture
218
8.1
Overview
218
8.2
Background and history
218
8.3
A user s perspective
219
8.4
Components
220
8.4.1
Runtime instance
222
8.4.2
Instance components
223
8.4.3
Instance backbone
227
8.4.4
Tooling
229
8.5
Services
232
8.5.1
Job management
232
8.5.2
Resource management and monitoring
236
8.5.3
Scheduling
239
8.5.4
Data transport
241
8.5.5
Fault tolerance
247
8.5.6
Logging, tracing, and error reporting
248
8.5.7
Security and access control
251
8.5.8
Application development support
256
8.5.9
Processing element
259
8.5.10
Debugging
264
8.5.11
Visualization
267
8.6
Concluding remarks
268
References
270
Part IV Application design and analytics
273
9
Design principles and patterns for stream processing applications
275
9.1
Overview
275
9.2
Functional design patterns and principles
275
9.2.1
Edge adaptation
275
9.2.2
Flow manipulation
287
9.2.3
Dynamic adaptation
301
9.3
Non-
functional principles and design patterns
310
9.3.1
Application design and composition
310
9.3.2
Parallelization
314
9.3.3
Performance optimization
325
9.3.4
Fault tolerance
333
Contents xi
9.4
Concluding remarks
339
References
339
10
Stream analytics: data pre-processing and transformation
342
IO.
I Overview
342
1
0.2
The mining process
342
1
0.3
Notation
344
10.4
Descriptive statistics
345
10.4.1
Illustrative technique: BasicCounting
348
10.4.2
Advanced reading
353
10.5
Sampling
353
10.5.1
Illustrative technique: reservoir sampling
356
10.5.2
Advanced reading
357
10.6
Sketches
358
10.6.1
Illustrative technique: Count-Min sketch
360
10.6.2
Advanced reading
363
10.7
Quantization
363
10.7.1
Illustrative techniques: binary clipping and moment preserving
quantization
366
10.7.2
Advanced reading
369
10.8
Dimensionality reduction
370
10.8.1
Illustrative technique: SPIRIT
373
10.8.2
Advanced reading
375
10.9
Transforms
375
10.9.1
Illustrative technique: the
Haar
transform
379
10.9.2
Advanced reading
383
10.10
Concluding remarks
383
References
383
11
Stream analytics: modeling and evaluation
388
11.1
Overview
388
11.2
Offline modeling and online evaluation
389
11.3
Data stream classification
394
11.3.1
Illustrative technique: VFDT
398
11.3.2
Advanced reading
402
11.4
Data stream clustering
403
11.4.1
Illustrative technique: CluStream microclustering
409
11.4.2
Advanced reading
413
11.5
Data stream regression
414
11.5.1
Illustrative technique: linear regression with SGD
417
11.5.2
Advanced reading
419
11.6
Data stream frequent pattern mining
420
11.6.1
Illustrative technique: lossy counting
425
11.6.2
Advanced reading
426
xii Contents
1
1.7
Anomaly detection
427
11.7.1
Illustrative technique: micro-clustering-based anomaly detection
432
11.7.2
Advanced reading
432
11.8
Concluding remarks
433
References
433
Part V Case studies
439
12
Applications
44
1
1
2.
1 Overview
44
1
1
2.2
The Operations Monitoring application
442
1
2.2.
1 Motivation
442
12.2.2
Requirements
443
12.2.3
Design
445
12.2.4
Analytics
451
12.2.5
Fault tolerance
453
12.3
The Patient Monitoring application
454
12.3.1
Motivation
454
12.3.2
Requirements
455
12.3.3
Design
456
12.3.4
Evaluation
463
12.4
The Semiconductor Process Control application
467
12.4.1
Motivation
467
12.4.2
Requirements
469
12.4.3
Design
472
12.4.4
Evaluation
479
12.4.5
User interface
481
12.5
Concluding remarks
482
References
482
Part VI Closing notes
485
13
Conclusion
487
13.1
Book summary
487
13.2
Challenges and open problems
488
13.2.1
Software engineering
488
13.2.2
Integration
491
13.2.3
Scaling up and distributed computing
493
13.2.4
Analytics
495
13.3
Where do we go from here?
496
References
497
Keywords and identifiers index
500
Index
504
|
| any_adam_object | 1 |
| author | Andrade, Henrique C. M. Gedik, Buǧra Turaga, Deepak S. |
| author_GND | (DE-588)1048709507 (DE-588)1048660338 (DE-588)1048660508 |
| author_facet | Andrade, Henrique C. M. Gedik, Buǧra Turaga, Deepak S. |
| author_role | aut aut aut |
| author_sort | Andrade, Henrique C. M. |
| author_variant | h c m a hcm hcma b g bg d s t ds dst |
| building | Verbundindex |
| bvnumber | BV041716948 |
| classification_rvk | ST 274 |
| ctrlnum | (OCoLC)871294049 (DE-599)HBZHT017848400 |
| dewey-full | 004.2/2 |
| dewey-hundreds | 000 - Computer science, information, general works |
| dewey-ones | 004 - Computer science |
| dewey-raw | 004.2/2 |
| dewey-search | 004.2/2 |
| dewey-sort | 14.2 12 |
| dewey-tens | 000 - Computer science, information, general works |
| discipline | Informatik |
| format | Book |
| fullrecord | <?xml version="1.0" encoding="UTF-8"?><collection xmlns="http://www.loc.gov/MARC21/slim"><record><leader>01760nam a22004092c 4500</leader><controlfield tag="001">BV041716948</controlfield><controlfield tag="003">DE-604</controlfield><controlfield tag="005">20160329 </controlfield><controlfield tag="007">t</controlfield><controlfield tag="008">140304s2014 ad|| |||| 00||| eng d</controlfield><datafield tag="016" ind1="7" ind2=" "><subfield code="a">016433827</subfield><subfield code="2">DE-101</subfield></datafield><datafield tag="020" ind1=" " ind2=" "><subfield code="a">9781107015548</subfield><subfield code="9">978-1-107-01554-8</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(OCoLC)871294049</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-599)HBZHT017848400</subfield></datafield><datafield tag="040" ind1=" " ind2=" "><subfield code="a">DE-604</subfield><subfield code="b">ger</subfield></datafield><datafield tag="041" ind1="0" ind2=" "><subfield code="a">eng</subfield></datafield><datafield tag="049" ind1=" " ind2=" "><subfield code="a">DE-473</subfield><subfield code="a">DE-83</subfield></datafield><datafield tag="082" ind1="0" ind2=" "><subfield code="a">004.2/2</subfield><subfield code="2">23</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">ST 274</subfield><subfield code="0">(DE-625)143641:</subfield><subfield code="2">rvk</subfield></datafield><datafield tag="100" ind1="1" ind2=" "><subfield code="a">Andrade, Henrique C. M.</subfield><subfield code="e">Verfasser</subfield><subfield code="0">(DE-588)1048709507</subfield><subfield code="4">aut</subfield></datafield><datafield tag="245" ind1="1" ind2="0"><subfield code="a">Fundamentals of stream processing</subfield><subfield code="b">application design, systems, and analytics</subfield><subfield code="c">Henrique C. M. Andrade, Buǧra Gedik, Deepak S. Turaga</subfield></datafield><datafield tag="264" ind1=" " ind2="1"><subfield code="a">Cambridge [u.a.]</subfield><subfield code="b">Cambridge Univ. Press</subfield><subfield code="c">2014</subfield></datafield><datafield tag="300" ind1=" " ind2=" "><subfield code="a">XXVII, 529 S.</subfield><subfield code="b">Ill., graph. Darst.</subfield></datafield><datafield tag="336" ind1=" " ind2=" "><subfield code="b">txt</subfield><subfield code="2">rdacontent</subfield></datafield><datafield tag="337" ind1=" " ind2=" "><subfield code="b">n</subfield><subfield code="2">rdamedia</subfield></datafield><datafield tag="338" ind1=" " ind2=" "><subfield code="b">nc</subfield><subfield code="2">rdacarrier</subfield></datafield><datafield tag="650" ind1="0" ind2="7"><subfield code="a">Hochleistungsrechnen</subfield><subfield code="0">(DE-588)4532701-4</subfield><subfield code="2">gnd</subfield><subfield code="9">rswk-swf</subfield></datafield><datafield tag="650" ind1="0" ind2="7"><subfield code="a">Streaming</subfield><subfield code="g">Kommunikationstechnik</subfield><subfield code="0">(DE-588)4614855-3</subfield><subfield code="2">gnd</subfield><subfield code="9">rswk-swf</subfield></datafield><datafield tag="653" ind1=" " ind2=" "><subfield code="a">Data flow computing.</subfield></datafield><datafield tag="653" ind1=" " ind2=" "><subfield code="a">SIMD (Computer architecture)</subfield></datafield><datafield tag="689" ind1="0" ind2="0"><subfield code="a">Hochleistungsrechnen</subfield><subfield code="0">(DE-588)4532701-4</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="0" ind2="1"><subfield code="a">Streaming</subfield><subfield code="g">Kommunikationstechnik</subfield><subfield code="0">(DE-588)4614855-3</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="0" ind2=" "><subfield code="5">DE-604</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">Gedik, Buǧra</subfield><subfield code="e">Verfasser</subfield><subfield code="0">(DE-588)1048660338</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">Turaga, Deepak S.</subfield><subfield code="e">Verfasser</subfield><subfield code="0">(DE-588)1048660508</subfield><subfield code="4">aut</subfield></datafield><datafield tag="856" ind1="4" ind2="2"><subfield code="m">Digitalisierung UB Bamberg - ADAM Catalogue Enrichment</subfield><subfield code="q">application/pdf</subfield><subfield code="u">http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=027164019&sequence=000002&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA</subfield><subfield code="3">Inhaltsverzeichnis</subfield></datafield><datafield tag="999" ind1=" " ind2=" "><subfield code="a">oai:aleph.bib-bvb.de:BVB01-027164019</subfield></datafield></record></collection> |
| id | DE-604.BV041716948 |
| illustrated | Illustrated |
| indexdate | 2024-07-10T01:03:38Z |
| institution | BVB |
| isbn | 9781107015548 |
| language | English |
| oai_aleph_id | oai:aleph.bib-bvb.de:BVB01-027164019 |
| oclc_num | 871294049 |
| open_access_boolean | |
| owner | DE-473 DE-BY-UBG DE-83 |
| owner_facet | DE-473 DE-BY-UBG DE-83 |
| physical | XXVII, 529 S. Ill., graph. Darst. |
| publishDate | 2014 |
| publishDateSearch | 2014 |
| publishDateSort | 2014 |
| publisher | Cambridge Univ. Press |
| record_format | marc |
| spelling | Andrade, Henrique C. M. Verfasser (DE-588)1048709507 aut Fundamentals of stream processing application design, systems, and analytics Henrique C. M. Andrade, Buǧra Gedik, Deepak S. Turaga Cambridge [u.a.] Cambridge Univ. Press 2014 XXVII, 529 S. Ill., graph. Darst. txt rdacontent n rdamedia nc rdacarrier Hochleistungsrechnen (DE-588)4532701-4 gnd rswk-swf Streaming Kommunikationstechnik (DE-588)4614855-3 gnd rswk-swf Data flow computing. SIMD (Computer architecture) Hochleistungsrechnen (DE-588)4532701-4 s Streaming Kommunikationstechnik (DE-588)4614855-3 s DE-604 Gedik, Buǧra Verfasser (DE-588)1048660338 aut Turaga, Deepak S. Verfasser (DE-588)1048660508 aut Digitalisierung UB Bamberg - ADAM Catalogue Enrichment application/pdf http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=027164019&sequence=000002&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA Inhaltsverzeichnis |
| spellingShingle | Andrade, Henrique C. M. Gedik, Buǧra Turaga, Deepak S. Fundamentals of stream processing application design, systems, and analytics Hochleistungsrechnen (DE-588)4532701-4 gnd Streaming Kommunikationstechnik (DE-588)4614855-3 gnd |
| subject_GND | (DE-588)4532701-4 (DE-588)4614855-3 |
| title | Fundamentals of stream processing application design, systems, and analytics |
| title_auth | Fundamentals of stream processing application design, systems, and analytics |
| title_exact_search | Fundamentals of stream processing application design, systems, and analytics |
| title_full | Fundamentals of stream processing application design, systems, and analytics Henrique C. M. Andrade, Buǧra Gedik, Deepak S. Turaga |
| title_fullStr | Fundamentals of stream processing application design, systems, and analytics Henrique C. M. Andrade, Buǧra Gedik, Deepak S. Turaga |
| title_full_unstemmed | Fundamentals of stream processing application design, systems, and analytics Henrique C. M. Andrade, Buǧra Gedik, Deepak S. Turaga |
| title_short | Fundamentals of stream processing |
| title_sort | fundamentals of stream processing application design systems and analytics |
| title_sub | application design, systems, and analytics |
| topic | Hochleistungsrechnen (DE-588)4532701-4 gnd Streaming Kommunikationstechnik (DE-588)4614855-3 gnd |
| topic_facet | Hochleistungsrechnen Streaming Kommunikationstechnik |
| url | http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=027164019&sequence=000002&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA |
| work_keys_str_mv | AT andradehenriquecm fundamentalsofstreamprocessingapplicationdesignsystemsandanalytics AT gedikbugra fundamentalsofstreamprocessingapplicationdesignsystemsandanalytics AT turagadeepaks fundamentalsofstreamprocessingapplicationdesignsystemsandanalytics |