Enterprise-specific In-Memory Data Management: hyrisec, an In-Memory column store engine for OLXP
Gespeichert in:
| 1. Verfasser: | |
|---|---|
| Format: | Abschlussarbeit Buch |
| Sprache: | English |
| Veröffentlicht: |
2014
|
| Schlagworte: | |
| Online-Zugang: | Inhaltsverzeichnis |
| Beschreibung: | IV, 201 S. graph. Darst. |
Internformat
MARC
| LEADER | 00000nam a2200000 c 4500 | ||
|---|---|---|---|
| 001 | BV041932522 | ||
| 003 | DE-604 | ||
| 005 | 20141001 | ||
| 007 | t | ||
| 008 | 140625s2014 d||| m||| 00||| eng d | ||
| 035 | |a (OCoLC)889842832 | ||
| 035 | |a (DE-599)GBV78679254X | ||
| 040 | |a DE-604 |b ger | ||
| 041 | 0 | |a eng | |
| 049 | |a DE-188 |a DE-12 | ||
| 082 | 0 | |a 005.758 |2 22/ger | |
| 100 | 1 | |a Krüger, Jens |d 197X- |e Verfasser |0 (DE-588)139082700 |4 aut | |
| 245 | 1 | 0 | |a Enterprise-specific In-Memory Data Management |b hyrisec, an In-Memory column store engine for OLXP |c von Jens Krüger |
| 264 | 1 | |c 2014 | |
| 300 | |a IV, 201 S. |b graph. Darst. | ||
| 336 | |b txt |2 rdacontent | ||
| 337 | |b n |2 rdamedia | ||
| 338 | |b nc |2 rdacarrier | ||
| 502 | |a Potsdam, Univ., Diss., 2014 | ||
| 546 | |a Mit einer Zssfassung in dt. Sprache | ||
| 650 | 0 | 7 | |a Datenbankentwurf |0 (DE-588)4127613-9 |2 gnd |9 rswk-swf |
| 655 | 7 | |0 (DE-588)4113937-9 |a Hochschulschrift |2 gnd-content | |
| 689 | 0 | 0 | |a Datenbankentwurf |0 (DE-588)4127613-9 |D s |
| 689 | 0 | |5 DE-604 | |
| 856 | 4 | 2 | |m DNB Datenaustausch |q application/pdf |u http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=027375788&sequence=000001&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA |3 Inhaltsverzeichnis |
| 999 | |a oai:aleph.bib-bvb.de:BVB01-027375788 | ||
Datensatz im Suchindex
| _version_ | 1804152307955269632 |
|---|---|
| adam_text | CONTENTS
1 INTRODUCTION 1
1.1 TRENDS IN ENTERPRISE COMPUTING 3
1.1.1 HARDWARE TRENDS 3
1.1.2 TODAYS ENTERPRISE APPLICATIONS 5
1.2 DISSERTATION GOALS. CHALLENGES, AND CONTRIBUTION 6
1.2.1 PROBLEM STATEMENT 6
1.2.2 RESEARCH QUESTIONS 8
1.2.3 THESIS OUTLINE AND SCIENTIFIC CONTRIBUTION 8
1.2.4 PUBLISHED RESEARCH RESULTS 11
1 ENTERPRISE-SPECIFIC DATA MANAGEMENT FOR MODERN HARDWARE 13
2 RELATED WORK 15
2.1 ONLINE MIXED WORKLOAD PROCESSING 15
2.1.1 ORIGINS OF MIXED WORKLOADS 15
2.1.2 APPLICATION-SPECIFIC DATABASES 17
2.1.3 WORKLOAD AND DATA CHARACTERISTICS 17
2.2 DATABASE OPTIMIZATIONS FOR MODERN HARDWARE 18
2.2.1 IN-MEMORY DATABASES 18
2.2.2 VERTICAL PARTITIONING 22
2.2.3 COMPRESSION TECHNIQUES IN DATABASES 24
2.2.4 PARALLEL PROCESSING FOR DATABASE OPERATIONS 25
2.2.5 COST MODELS FOR IN-MEMORY DATABASE SYSTEMS 26
2.3 UPDATES AND LOOKUPS IN READ-OPTIMIZED DATABASES 28
2.3.1 DIFFERENTIAL STORE
1
28
2.3.2 PARALLEL ATTRIBUTE MERGE 31
2.3.3 INDICES FOR IN-MEMORY DATABASES 31
2.4 SUM MARJ 33
3 OLXP * A NOVEL WORKLOAD CATEGORY FOR ENTERPRISE DATA MANAGEMENT 35
3.1 MOTIVATION 35
3.2 BACKGROUND AND METHODOLOGY 36
3.2.1 ENTERPRISE APPLICATION CLASSIFICATION 37
I
HTTP://D-NB.INFO/1054344175
CONTENTS
3.2.2 SCENARIO-BASED SELECTION OF ENTERPRISE APPLICATIONS 38
3.2.3 APPLICATION CHARACTERISTICS AS METHODOLOGY 38
3.3 REQUIREMENTS DEFINED BY APPLICATION CHARACTERISTICS 42
3.3.1 APPLICATION CHARACTERISTICS OF ENTERPRISE APPLICATIONS 42
3.3.2 APPLICATION CHARACTERISTICS COMPARISON 44
3.4 REQUIREMENTS DEFINED BY WORKLOADS AND DATA 46
3.4.1 DATABASE WORKLOADS IN ENTERPRISE APPLICATIONS 46
3.4.2 DATASET AND DATA SCHEMA CHARACTERISTICS 47
3.5 ONLINE MIXED WORKLOAD PROCESSING (OLXP) 51
3.5.1 CHARACTERISTICS OF ONLINE TRANSACTION PROCESSING (OLTP) 51
3.5.2 CHARACTERISTICS OF ONLINE ANALYTICAL PROCESSING (OLAP) 52
3.5.3 CHARACTERISTICS OF ONLINE MIXED WORKLOAD PROCESSING (OLXP) .......
52
3.6 CONCLUSIONS 53
4 DATA PROCESSING AND MANAGEMENT ON MODERN HARDWARE 55
4.1 MOTIVATION 56
4.2 BRIEF SUMMARY OF MAIN MEMORY ACCESS ON MODERN HARDWARE 56
4.2.1 DATA ACCESS ON THE MEMORY HIERARCHY 57
4.2.2 CACHE-CONSCIOUS DATA PROCESSING 60
4.3 STORAGE TECHNIQUES FOR MODERN HARDWARE 63
4.3.1 IN-MEMORY DATABASE 63
4.3.2 COLUMN-ORIENTED PHYSICAL DATA ORGANIZATION 64
4.3.3 IN-MEMORY COLUMN STORE 65
4.3.4 QUERY EXECUTION IN IN-MEMORY COLUMN STORES 67
4.3.5 INSERT ONLY DATA MANAGEMENT 69
4.3.6 DATA COMPRESSION IN IN-MEMORY COLUMN STORES 69
4.3.7 PARALLEL PROCESSING FOR DATABASE OPERATIONS 72
4.4 CONCLUSIONS 73
II HYRISE
C
* AN IN-MEMORY COLUMN STORE ENGINE FOR OLXP 75
5 HYRISE
C
OVERVIEW 77
5.1 MOTIVATION 77
5.2 SYSTEM OVERVIEW AND TERMINOLOGY 78
5.2.1 OVERVIEW 78
5.2.2 IN-MEMORY STORAGE ENGINE OF HYRISE
C
79
5.2.3 TERMINOLOGY 82
5.2.4 PHYSICAL COLUMN ORGANIZATION IN HYRISE
0
83
5.3 QUERY EXECUTION IN HYRISE
C
84
5.3.1 OPERATIONS ON COLUMNS 85
5.3.2 ELEMENTARY DATABASE OPERATORS IN HYRISE
C
87
5.3.3 FURTHER OPERATORS IN HYRISE
C
90
5.3.4 OPERATOR PARALLELISM IN HYRISE
0
91
CONTENTS
5.4 PHYSICAL COLUMN DESIGN CONSIDERATIONS 93
5.4.1 TABLE CARDINALITY INFLUENCE ON OPERATORS 94
5.4.2 DISTINCT VALUE CARDINALITY INFLUENCE ON OPERATORS 95
5.4.3 PHYSICAL COLUMN REPRESENTATION IN HYRISE
C
96
5.5 CONCLUSIONS 98
6 DIFFERENTIAL STORE * WRITE PERFORMANCE FOR READ*OPTIMIZED DATABASES 99
6.1 MOTIVATION 99
6.2 INTRODUCING THE DIFFERENTIAL STORE 100
6.2.1 DATA CONSISTENCY 101
6.2.2 PARTITIONED COLUMNS 102
6.2.3 QUERY EXECUTION USING THE DIFFERENTIAL STORE 102
6.3 ONLINE MERGE PROCESS 103
6.3.1 PHASES OF THE ONLINE MERGE PROCESS 104
6.3.2 ONLINE MERGE PROCESS LOCK GRANULARITY 106
6.3.3 FAILURE CASE AND RECOVERY 106
6.3.4 ONLINE MERGE PROCESS IMPACT ON QUERY PROCESSING 106
6.3.5 TRIGGER STRATEGIES FOR THE ONLINE MERGE PROCESS 107
6.4 CONCLUSIONS 109
7 GROUP-KEY INDEX * FAST LOOKUPS FOR READ*OPTIMIZED DATABASES 111
7.1 MOTIVATION ILL
7.1.1 DESIGN CONSIDERATIONS 112
7.1.2 INDICES FOR HYRISE
C
112
7.2 A MAIN PARTITION COLUMN INDEX: GROUP-KEY INDEX 114
7.2.1 GROUP-KEY INDEX STRUCTURE 114
7.2.2 COMPRESSION OF THE GROUP-KEY INDEX 116
7.3 ANALYTICAL CONSIDERATION FOR READ ACCESS 117
7.3.1 COLUMN EQUAL SCAN COSTS 117
7.3.2 GROUP-KEY INDEX READ COSTS 118
7.3.3 BREAK EVEN POINT: INDEX LOOKUPS VS. COLUMN SCANS 120
7.4 GROUP-KEY INDEX EVALUATION 121
7.5 DELTA PARTITION CSB+ INDEX 122
7.6 CONCLUSIONS 123
III ATTRIBUTE MERGE * MAINTENANCE FOR COMPRESSED COLUMNS AND
INDICES 125
8 PARALLEL ATTRIBUTE MERGE 127
8.1 MOTIVATION 127
8.1.1 BOTTLENECK ATTRIBUTE MERGE 127
8.1.2 UPDATE RATE 129
8.2 ATTRIBUTE MERGE ALGORITHM 129
III
CONTENTS
8.2.1 AN EXAMPLE: MERGING MAIN AND DELTA PARTITION 129
8.2.2 DEFINING THE ATTRIBUTE MERGE ALGORITHM 130
8.2.3 MERGING DICTIONARIES (STEP 1) 131
8.2.4 UPDATING COMPRESSED VALUES (STEP 2) 132
8.2.5 OPTIMIZED ATTRIBUTE MERGE ALGORITHM 133
8.3 PARALLEL ATTRIBUTE MERGE IMPLEMENTATION 136
8.3.1 SCALAR IMPLEMENTATION 137
8.3.2 EXPLOITING THREAD-LEVEL PARALLELISM 139
8.4 PERFORMANCE EVALUATION 143
8.4.1 IMPACT OF DELTA PARTITION SIZE 145
8.4.2 IMPACT OF VALUE LENGTH AND PERCENTAGE OF UNIQUE VALUES 146
8.4.3 PARALLEL SCALABILITY AND PERCENTAGE OF UNIQUE VALUES 147
8.4.4 IMPACT OF MAIN PARTITION SIZE AND PERCENTAGE OF UNIQUE ENTRIES 148
8.4.5 COMPARISON WITH ANALYTICAL MODEL 149
8.5 CONCLUSIONS 151
9 INDEX-AWARE PARALLEL ATTRIBUTE MERGE 153
9.1 MOTIVATION 153
9.2 GROUP-KEY INDEX REBUILD AFTER PARALLEL ATTRIBUTE MERGE 154
9.3 GROUP-KEY INDEX-AWARE PARALLEL ATTRIBUTE MERGE 155
9.4 EVALUATION 157
9.4.1 BREAK-EVEN POINT CONSIDERATION 157
9.4.2 MEMORY TRAFFIC CONSIDERATION 159
9.4.3 MEASUREMENT 160
9.5 CONCLUSIONS 161
IV CONCLUSIONS 163
10 SUMMARY 165
10.1 CONCLUSIONS 165
10.2 FUTURE WORK 167
10.2.1 ONLINE MERGE PROCESS SCHEDULING 167
10.2.2 SHARED-DICTIONARY JOINS 167
10.2.3 DICTIONARIES AS FOREIGN-KEY TABLES 168
V APPENDIX 171
A SYMBOL DEFINITION 173
BIBLIOGRAPHY 201
IV
|
| any_adam_object | 1 |
| author | Krüger, Jens 197X- |
| author_GND | (DE-588)139082700 |
| author_facet | Krüger, Jens 197X- |
| author_role | aut |
| author_sort | Krüger, Jens 197X- |
| author_variant | j k jk |
| building | Verbundindex |
| bvnumber | BV041932522 |
| ctrlnum | (OCoLC)889842832 (DE-599)GBV78679254X |
| dewey-full | 005.758 |
| dewey-hundreds | 000 - Computer science, information, general works |
| dewey-ones | 005 - Computer programming, programs, data, security |
| dewey-raw | 005.758 |
| dewey-search | 005.758 |
| dewey-sort | 15.758 |
| dewey-tens | 000 - Computer science, information, general works |
| discipline | Informatik |
| format | Thesis Book |
| fullrecord | <?xml version="1.0" encoding="UTF-8"?><collection xmlns="http://www.loc.gov/MARC21/slim"><record><leader>01307nam a2200337 c 4500</leader><controlfield tag="001">BV041932522</controlfield><controlfield tag="003">DE-604</controlfield><controlfield tag="005">20141001 </controlfield><controlfield tag="007">t</controlfield><controlfield tag="008">140625s2014 d||| m||| 00||| eng d</controlfield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(OCoLC)889842832</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-599)GBV78679254X</subfield></datafield><datafield tag="040" ind1=" " ind2=" "><subfield code="a">DE-604</subfield><subfield code="b">ger</subfield></datafield><datafield tag="041" ind1="0" ind2=" "><subfield code="a">eng</subfield></datafield><datafield tag="049" ind1=" " ind2=" "><subfield code="a">DE-188</subfield><subfield code="a">DE-12</subfield></datafield><datafield tag="082" ind1="0" ind2=" "><subfield code="a">005.758</subfield><subfield code="2">22/ger</subfield></datafield><datafield tag="100" ind1="1" ind2=" "><subfield code="a">Krüger, Jens</subfield><subfield code="d">197X-</subfield><subfield code="e">Verfasser</subfield><subfield code="0">(DE-588)139082700</subfield><subfield code="4">aut</subfield></datafield><datafield tag="245" ind1="1" ind2="0"><subfield code="a">Enterprise-specific In-Memory Data Management</subfield><subfield code="b">hyrisec, an In-Memory column store engine for OLXP</subfield><subfield code="c">von Jens Krüger</subfield></datafield><datafield tag="264" ind1=" " ind2="1"><subfield code="c">2014</subfield></datafield><datafield tag="300" ind1=" " ind2=" "><subfield code="a">IV, 201 S.</subfield><subfield code="b">graph. Darst.</subfield></datafield><datafield tag="336" ind1=" " ind2=" "><subfield code="b">txt</subfield><subfield code="2">rdacontent</subfield></datafield><datafield tag="337" ind1=" " ind2=" "><subfield code="b">n</subfield><subfield code="2">rdamedia</subfield></datafield><datafield tag="338" ind1=" " ind2=" "><subfield code="b">nc</subfield><subfield code="2">rdacarrier</subfield></datafield><datafield tag="502" ind1=" " ind2=" "><subfield code="a">Potsdam, Univ., Diss., 2014</subfield></datafield><datafield tag="546" ind1=" " ind2=" "><subfield code="a">Mit einer Zssfassung in dt. Sprache</subfield></datafield><datafield tag="650" ind1="0" ind2="7"><subfield code="a">Datenbankentwurf</subfield><subfield code="0">(DE-588)4127613-9</subfield><subfield code="2">gnd</subfield><subfield code="9">rswk-swf</subfield></datafield><datafield tag="655" ind1=" " ind2="7"><subfield code="0">(DE-588)4113937-9</subfield><subfield code="a">Hochschulschrift</subfield><subfield code="2">gnd-content</subfield></datafield><datafield tag="689" ind1="0" ind2="0"><subfield code="a">Datenbankentwurf</subfield><subfield code="0">(DE-588)4127613-9</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="0" ind2=" "><subfield code="5">DE-604</subfield></datafield><datafield tag="856" ind1="4" ind2="2"><subfield code="m">DNB Datenaustausch</subfield><subfield code="q">application/pdf</subfield><subfield code="u">http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=027375788&sequence=000001&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA</subfield><subfield code="3">Inhaltsverzeichnis</subfield></datafield><datafield tag="999" ind1=" " ind2=" "><subfield code="a">oai:aleph.bib-bvb.de:BVB01-027375788</subfield></datafield></record></collection> |
| genre | (DE-588)4113937-9 Hochschulschrift gnd-content |
| genre_facet | Hochschulschrift |
| id | DE-604.BV041932522 |
| illustrated | Illustrated |
| indexdate | 2024-07-10T01:08:38Z |
| institution | BVB |
| language | English |
| oai_aleph_id | oai:aleph.bib-bvb.de:BVB01-027375788 |
| oclc_num | 889842832 |
| open_access_boolean | |
| owner | DE-188 DE-12 |
| owner_facet | DE-188 DE-12 |
| physical | IV, 201 S. graph. Darst. |
| publishDate | 2014 |
| publishDateSearch | 2014 |
| publishDateSort | 2014 |
| record_format | marc |
| spelling | Krüger, Jens 197X- Verfasser (DE-588)139082700 aut Enterprise-specific In-Memory Data Management hyrisec, an In-Memory column store engine for OLXP von Jens Krüger 2014 IV, 201 S. graph. Darst. txt rdacontent n rdamedia nc rdacarrier Potsdam, Univ., Diss., 2014 Mit einer Zssfassung in dt. Sprache Datenbankentwurf (DE-588)4127613-9 gnd rswk-swf (DE-588)4113937-9 Hochschulschrift gnd-content Datenbankentwurf (DE-588)4127613-9 s DE-604 DNB Datenaustausch application/pdf http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=027375788&sequence=000001&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA Inhaltsverzeichnis |
| spellingShingle | Krüger, Jens 197X- Enterprise-specific In-Memory Data Management hyrisec, an In-Memory column store engine for OLXP Datenbankentwurf (DE-588)4127613-9 gnd |
| subject_GND | (DE-588)4127613-9 (DE-588)4113937-9 |
| title | Enterprise-specific In-Memory Data Management hyrisec, an In-Memory column store engine for OLXP |
| title_auth | Enterprise-specific In-Memory Data Management hyrisec, an In-Memory column store engine for OLXP |
| title_exact_search | Enterprise-specific In-Memory Data Management hyrisec, an In-Memory column store engine for OLXP |
| title_full | Enterprise-specific In-Memory Data Management hyrisec, an In-Memory column store engine for OLXP von Jens Krüger |
| title_fullStr | Enterprise-specific In-Memory Data Management hyrisec, an In-Memory column store engine for OLXP von Jens Krüger |
| title_full_unstemmed | Enterprise-specific In-Memory Data Management hyrisec, an In-Memory column store engine for OLXP von Jens Krüger |
| title_short | Enterprise-specific In-Memory Data Management |
| title_sort | enterprise specific in memory data management hyrisec an in memory column store engine for olxp |
| title_sub | hyrisec, an In-Memory column store engine for OLXP |
| topic | Datenbankentwurf (DE-588)4127613-9 gnd |
| topic_facet | Datenbankentwurf Hochschulschrift |
| url | http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=027375788&sequence=000001&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA |
| work_keys_str_mv | AT krugerjens enterprisespecificinmemorydatamanagementhyrisecaninmemorycolumnstoreengineforolxp |