Natural language processing with transformers: building language applications with hugging face
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Buch |
| Sprache: | English |
| Veröffentlicht: |
Beijing ; Boston ; Farnham ; Sebastopol ; Tokyo
O'Reilly
May 2022
|
| Ausgabe: | Revised edition |
| Schlagworte: | |
| Online-Zugang: | Inhaltsverzeichnis |
| Beschreibung: | xxii, 383 Seiten Illustrationen, Diagramme (farbig) |
| ISBN: | 9781098136796 1098136799 |
Internformat
MARC
| LEADER | 00000nam a2200000 c 4500 | ||
|---|---|---|---|
| 001 | BV048312737 | ||
| 003 | DE-604 | ||
| 005 | 20241219 | ||
| 007 | t| | ||
| 008 | 220705s2022 xx a||| |||| 00||| eng d | ||
| 020 | |a 9781098136796 |c paperback: circa EUR 64.00 (DE) |9 978-1-098-13679-6 | ||
| 020 | |a 1098136799 |9 1-0981-3679-9 | ||
| 035 | |a (OCoLC)1304482473 | ||
| 035 | |a (DE-599)BVBBV048312737 | ||
| 040 | |a DE-604 |b ger |e rda | ||
| 041 | 0 | |a eng | |
| 049 | |a DE-92 |a DE-898 |a DE-858 |a DE-Aug4 |a DE-11 |a DE-1050 |a DE-523 |a DE-473 | ||
| 084 | |a ZN 6070 |0 (DE-625)157501: |2 rvk | ||
| 084 | |a ST 306 |0 (DE-625)143654: |2 rvk | ||
| 084 | |a PI 4120 |0 (DE-625)136625: |2 rvk | ||
| 100 | 1 | |a Tunstall, Lewis |e Verfasser |0 (DE-588)1254023267 |4 aut | |
| 245 | 1 | 0 | |a Natural language processing with transformers |b building language applications with hugging face |c Lewis Tunstall, Leandro von Werra, and Thomas Wolf ; foreword by Aurélien Géron |
| 250 | |a Revised edition | ||
| 264 | 1 | |a Beijing ; Boston ; Farnham ; Sebastopol ; Tokyo |b O'Reilly |c May 2022 | |
| 300 | |a xxii, 383 Seiten |b Illustrationen, Diagramme (farbig) | ||
| 336 | |b txt |2 rdacontent | ||
| 337 | |b n |2 rdamedia | ||
| 338 | |b nc |2 rdacarrier | ||
| 650 | 0 | 7 | |a Deep Learning |0 (DE-588)1135597375 |2 gnd |9 rswk-swf |
| 650 | 0 | 7 | |a Natürliche Sprache |0 (DE-588)4041354-8 |2 gnd |9 rswk-swf |
| 650 | 0 | 7 | |a Textverarbeitung |0 (DE-588)4059667-9 |2 gnd |9 rswk-swf |
| 650 | 0 | 7 | |a Sprachverarbeitung |0 (DE-588)4116579-2 |2 gnd |9 rswk-swf |
| 650 | 0 | 7 | |a Maschinelles Lernen |0 (DE-588)4193754-5 |2 gnd |9 rswk-swf |
| 689 | 0 | 0 | |a Natürliche Sprache |0 (DE-588)4041354-8 |D s |
| 689 | 0 | 1 | |a Sprachverarbeitung |0 (DE-588)4116579-2 |D s |
| 689 | 0 | 2 | |a Textverarbeitung |0 (DE-588)4059667-9 |D s |
| 689 | 0 | 3 | |a Maschinelles Lernen |0 (DE-588)4193754-5 |D s |
| 689 | 0 | 4 | |a Deep Learning |0 (DE-588)1135597375 |D s |
| 689 | 0 | |5 DE-604 | |
| 700 | 1 | |a Werra, Leandro von |e Verfasser |0 (DE-588)1254024301 |4 aut | |
| 700 | 1 | |a Wolf, Thomas |e Verfasser |0 (DE-588)1280300140 |4 aut | |
| 700 | 1 | |a Géron, Aurélien |0 (DE-588)1131560930 |4 wpr | |
| 856 | 4 | 2 | |m Digitalisierung UB Bamberg - ADAM Catalogue Enrichment |q application/pdf |u http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=033692262&sequence=000001&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA |3 Inhaltsverzeichnis |
| 943 | 1 | |a oai:aleph.bib-bvb.de:BVB01-033692262 | |
Datensatz im Suchindex
| _version_ | 1823932150225829888 |
|---|---|
| adam_text |
Table of Contents Foreword. xi Preface. xv 1. Hello Transformers. 1 The Encoder-Decoder Framework Attention Mechanisms Transfer Learning in NLP Hugging Face Transformers: Bridging the Gap A Tour of Transformer Applications Text Classification Named Entity Recognition Question Answering Summarization Translation Text Generation The Hugging Face Ecosystem The Hugging Face Hub Hugging Face Tokenizers Hugging Face Datasets Hugging Face Accelerate Main Challenges with Transformers Conclusion 2 4 6 9 10 10 11 12 13 13 14 15 16 17 18 18 19 20 2. Text Classification. 21 The Dataset A First Look at Hugging Face Datasets From Datasets to DataFrames 22 23 26 V
Looking at the Class Distribution How Long Are Our Tweets? From Text to Tokens Character Tokenization Word Tokenization Subword Tokenization Tokenizing the Whole Dataset Training a Text Classifier Transformers as Feature Extractors Fine-Tuning Transformers Conclusion 27 28 29 29 31 33 35 36 38 45 54 3. Transformer Anatomy. 57 The Transformer Architecture The Encoder Self-Attention The Feed-Forward Layer Adding Layer Normalization Positional Embeddings Adding a Classification Head The Decoder Meet the Transformers The Transformer Tree of Life The Encoder Branch The Decoder Branch The Encoder-Decoder Branch Conclusion 57 60 61 70 71 73 75 76 78 78 79 82 83 85 4. Multilingual Named Entity Recognition. 87 The Dataset Multilingual Transformers A Closer Look at Tokenization The Tokenizer Pipeline The SentencePiece Tokenizer Transformers for Named Entity Recognition The Anatomy of the Transformers Model Class Bodies and Heads Creating a Custom Model for Token Classification Loading a Custom Model Tokenizing Texts for NER Performance Measures Fine-Tuning XLM-RoBERTa vi I Table of Contents 88 92 93 94 95 96 98 98 99 101 103 105 106
Error Analysis Cross-Lingual Transfer When Does Zero-Shot Transfer Make Sense? Fine-Tuning on Multiple Languages at Once Interacting with Model Widgets Conclusion 108 115 116 118 121 122 5. Text Generation. 123 The Challenge with Generating Coherent Text Greedy Search Decoding Beam Search Decoding Sampling Methods Top-к and Nucleus Sampling Which Decoding Method Is Best? Conclusion 125 127 130 134 136 140 140 6. Summarization. 141 The CNN/DailyMail Dataset Text Summarization Pipelines Summarization Baseline GPT-2 T5 BART PEGASUS Comparing Different Summaries Measuring the Quality of Generated Text BLEU ROUGE Evaluating PEGASUS on the CNN/DailyMail Dataset Training a Summarization Model Evaluating PEGASUS on SAMSum Fine-Tuning PEGASUS Generating Dialogue Summaries Conclusion 141 143 143 144 144 145 145 146 148 148 152 154 157 158 158 162 163 7. Question Answering. 165 Building a Review-Based QA System The Dataset Extracting Answers from Text Using Haystack to Build a QA Pipeline Improving Our QA Pipeline Evaluating the Retriever 166 167 173 181 189 189 Table of Contents | vii
Evaluating the Reader Domain Adaptation Evaluating the Whole QA Pipeline Going Beyond Extractive QA Conclusion 196 199 203 205 207 8. Making Transformers Efficient in Production. 209 Intent Detection as a Case Study Creating a Performance Benchmark Making Models Smaller via Knowledge Distillation Knowledge Distillation for Fine-Tuning Knowledge Distillation for Pretraining Creating a Knowledge Distillation Trainer Choosing a Good Student Initialization Finding Good Hyperparameters with Optuna Benchmarking Our Distilled Model Making Models Faster with Quantization Benchmarking Our Quantized Model Optimizing Inference with ONNX and the ONNX Runtime Making Models Sparser with Weight Pruning Sparsity in Deep Neural Networks Weight Pruning Methods Conclusion 210 212 217 217 220 220 222 226 229 230 236 237 243 244 244 248 9. Dealing with Few to No Labels. 249 Building a GitHub Issues Tagger Getting the Data Preparing the Data Creating Training Sets Creating Training Slices Implementing a Naive Bayesline Working with No Labeled Data Working with a Few Labels Data Augmentation Using Embeddings as a Lookup Table Fine-Tuning a Vanilla Transformer In-Context and Few-Shot Learning with Prompts Leveraging Unlabeled Data Fine-Tuning a Language Model Fine-Tuning a Classifier Advanced Methods Conclusion viii I Table of Contents 251 252 253 257 259 260 263 271 271 275 284 288 289 289 293 295 297
10. Training Transformers from Scratch. 299 00000000000000 Large Datasets and Where to Find Them Challenges of Building a Large-Scale Corpus Building a Custom Code Dataset Working with Large Datasets Adding Datasets to the Hugging Face Hub Building a Tokenizer The Tokenizer Model Measuring Tokenizer Performance A Tokenizer for Python Training a Tokenizer Saving a Custom Tokenizer on the Hub Training a Model from Scratch A Tale of Pretraining Objectives Initializing the Model Implementing the Dataloader Defining the Training Loop The Training Run Results and Analysis Conclusion 300 300 303 306 309 310 312 312 313 318 322 323 323 325 326 330 337 338 343 11. Future Directions. 345 Scaling Transformers Scaling Laws Challenges with Scaling Attention Please! Sparse Attention Linearized Attention Going Beyond Text Vision Tables Multimodal Transformers Speech-to-Text Vision and Text Where to from Here? 345 347 349 351 352 353 354 355 359 361 361 364 370 Index. 371 Table of Contents | ix |
| adam_txt | |
| any_adam_object | 1 |
| any_adam_object_boolean | |
| author | Tunstall, Lewis Werra, Leandro von Wolf, Thomas |
| author_GND | (DE-588)1254023267 (DE-588)1254024301 (DE-588)1280300140 (DE-588)1131560930 |
| author_facet | Tunstall, Lewis Werra, Leandro von Wolf, Thomas |
| author_role | aut aut aut |
| author_sort | Tunstall, Lewis |
| author_variant | l t lt l v w lv lvw t w tw |
| building | Verbundindex |
| bvnumber | BV048312737 |
| classification_rvk | ZN 6070 ST 306 PI 4120 |
| ctrlnum | (OCoLC)1304482473 (DE-599)BVBBV048312737 |
| discipline | Rechtswissenschaft Informatik Elektrotechnik / Elektronik / Nachrichtentechnik |
| discipline_str_mv | Rechtswissenschaft Informatik Elektrotechnik / Elektronik / Nachrichtentechnik |
| edition | Revised edition |
| format | Book |
| fullrecord | <?xml version="1.0" encoding="UTF-8"?><collection xmlns="http://www.loc.gov/MARC21/slim"><record><leader>00000nam a2200000 c 4500</leader><controlfield tag="001">BV048312737</controlfield><controlfield tag="003">DE-604</controlfield><controlfield tag="005">20241219</controlfield><controlfield tag="007">t|</controlfield><controlfield tag="008">220705s2022 xx a||| |||| 00||| eng d</controlfield><datafield tag="020" ind1=" " ind2=" "><subfield code="a">9781098136796</subfield><subfield code="c">paperback: circa EUR 64.00 (DE)</subfield><subfield code="9">978-1-098-13679-6</subfield></datafield><datafield tag="020" ind1=" " ind2=" "><subfield code="a">1098136799</subfield><subfield code="9">1-0981-3679-9</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(OCoLC)1304482473</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-599)BVBBV048312737</subfield></datafield><datafield tag="040" ind1=" " ind2=" "><subfield code="a">DE-604</subfield><subfield code="b">ger</subfield><subfield code="e">rda</subfield></datafield><datafield tag="041" ind1="0" ind2=" "><subfield code="a">eng</subfield></datafield><datafield tag="049" ind1=" " ind2=" "><subfield code="a">DE-92</subfield><subfield code="a">DE-898</subfield><subfield code="a">DE-858</subfield><subfield code="a">DE-Aug4</subfield><subfield code="a">DE-11</subfield><subfield code="a">DE-1050</subfield><subfield code="a">DE-523</subfield><subfield code="a">DE-473</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">ZN 6070</subfield><subfield code="0">(DE-625)157501:</subfield><subfield code="2">rvk</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">ST 306</subfield><subfield code="0">(DE-625)143654:</subfield><subfield code="2">rvk</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">PI 4120</subfield><subfield code="0">(DE-625)136625:</subfield><subfield code="2">rvk</subfield></datafield><datafield tag="100" ind1="1" ind2=" "><subfield code="a">Tunstall, Lewis</subfield><subfield code="e">Verfasser</subfield><subfield code="0">(DE-588)1254023267</subfield><subfield code="4">aut</subfield></datafield><datafield tag="245" ind1="1" ind2="0"><subfield code="a">Natural language processing with transformers</subfield><subfield code="b">building language applications with hugging face</subfield><subfield code="c">Lewis Tunstall, Leandro von Werra, and Thomas Wolf ; foreword by Aurélien Géron</subfield></datafield><datafield tag="250" ind1=" " ind2=" "><subfield code="a">Revised edition</subfield></datafield><datafield tag="264" ind1=" " ind2="1"><subfield code="a">Beijing ; Boston ; Farnham ; Sebastopol ; Tokyo</subfield><subfield code="b">O'Reilly</subfield><subfield code="c">May 2022</subfield></datafield><datafield tag="300" ind1=" " ind2=" "><subfield code="a">xxii, 383 Seiten</subfield><subfield code="b">Illustrationen, Diagramme (farbig)</subfield></datafield><datafield tag="336" ind1=" " ind2=" "><subfield code="b">txt</subfield><subfield code="2">rdacontent</subfield></datafield><datafield tag="337" ind1=" " ind2=" "><subfield code="b">n</subfield><subfield code="2">rdamedia</subfield></datafield><datafield tag="338" ind1=" " ind2=" "><subfield code="b">nc</subfield><subfield code="2">rdacarrier</subfield></datafield><datafield tag="650" ind1="0" ind2="7"><subfield code="a">Deep Learning</subfield><subfield code="0">(DE-588)1135597375</subfield><subfield code="2">gnd</subfield><subfield code="9">rswk-swf</subfield></datafield><datafield tag="650" ind1="0" ind2="7"><subfield code="a">Natürliche Sprache</subfield><subfield code="0">(DE-588)4041354-8</subfield><subfield code="2">gnd</subfield><subfield code="9">rswk-swf</subfield></datafield><datafield tag="650" ind1="0" ind2="7"><subfield code="a">Textverarbeitung</subfield><subfield code="0">(DE-588)4059667-9</subfield><subfield code="2">gnd</subfield><subfield code="9">rswk-swf</subfield></datafield><datafield tag="650" ind1="0" ind2="7"><subfield code="a">Sprachverarbeitung</subfield><subfield code="0">(DE-588)4116579-2</subfield><subfield code="2">gnd</subfield><subfield code="9">rswk-swf</subfield></datafield><datafield tag="650" ind1="0" ind2="7"><subfield code="a">Maschinelles Lernen</subfield><subfield code="0">(DE-588)4193754-5</subfield><subfield code="2">gnd</subfield><subfield code="9">rswk-swf</subfield></datafield><datafield tag="689" ind1="0" ind2="0"><subfield code="a">Natürliche Sprache</subfield><subfield code="0">(DE-588)4041354-8</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="0" ind2="1"><subfield code="a">Sprachverarbeitung</subfield><subfield code="0">(DE-588)4116579-2</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="0" ind2="2"><subfield code="a">Textverarbeitung</subfield><subfield code="0">(DE-588)4059667-9</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="0" ind2="3"><subfield code="a">Maschinelles Lernen</subfield><subfield code="0">(DE-588)4193754-5</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="0" ind2="4"><subfield code="a">Deep Learning</subfield><subfield code="0">(DE-588)1135597375</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="0" ind2=" "><subfield code="5">DE-604</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">Werra, Leandro von</subfield><subfield code="e">Verfasser</subfield><subfield code="0">(DE-588)1254024301</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">Wolf, Thomas</subfield><subfield code="e">Verfasser</subfield><subfield code="0">(DE-588)1280300140</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">Géron, Aurélien</subfield><subfield code="0">(DE-588)1131560930</subfield><subfield code="4">wpr</subfield></datafield><datafield tag="856" ind1="4" ind2="2"><subfield code="m">Digitalisierung UB Bamberg - ADAM Catalogue Enrichment</subfield><subfield code="q">application/pdf</subfield><subfield code="u">http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=033692262&sequence=000001&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA</subfield><subfield code="3">Inhaltsverzeichnis</subfield></datafield><datafield tag="943" ind1="1" ind2=" "><subfield code="a">oai:aleph.bib-bvb.de:BVB01-033692262</subfield></datafield></record></collection> |
| id | DE-604.BV048312737 |
| illustrated | Illustrated |
| index_date | 2024-07-03T20:10:08Z |
| indexdate | 2025-02-13T09:00:46Z |
| institution | BVB |
| isbn | 9781098136796 1098136799 |
| language | English |
| oai_aleph_id | oai:aleph.bib-bvb.de:BVB01-033692262 |
| oclc_num | 1304482473 |
| open_access_boolean | |
| owner | DE-92 DE-898 DE-BY-UBR DE-858 DE-Aug4 DE-11 DE-1050 DE-523 DE-473 DE-BY-UBG |
| owner_facet | DE-92 DE-898 DE-BY-UBR DE-858 DE-Aug4 DE-11 DE-1050 DE-523 DE-473 DE-BY-UBG |
| physical | xxii, 383 Seiten Illustrationen, Diagramme (farbig) |
| publishDate | 2022 |
| publishDateSearch | 2022 |
| publishDateSort | 2022 |
| publisher | O'Reilly |
| record_format | marc |
| spelling | Tunstall, Lewis Verfasser (DE-588)1254023267 aut Natural language processing with transformers building language applications with hugging face Lewis Tunstall, Leandro von Werra, and Thomas Wolf ; foreword by Aurélien Géron Revised edition Beijing ; Boston ; Farnham ; Sebastopol ; Tokyo O'Reilly May 2022 xxii, 383 Seiten Illustrationen, Diagramme (farbig) txt rdacontent n rdamedia nc rdacarrier Deep Learning (DE-588)1135597375 gnd rswk-swf Natürliche Sprache (DE-588)4041354-8 gnd rswk-swf Textverarbeitung (DE-588)4059667-9 gnd rswk-swf Sprachverarbeitung (DE-588)4116579-2 gnd rswk-swf Maschinelles Lernen (DE-588)4193754-5 gnd rswk-swf Natürliche Sprache (DE-588)4041354-8 s Sprachverarbeitung (DE-588)4116579-2 s Textverarbeitung (DE-588)4059667-9 s Maschinelles Lernen (DE-588)4193754-5 s Deep Learning (DE-588)1135597375 s DE-604 Werra, Leandro von Verfasser (DE-588)1254024301 aut Wolf, Thomas Verfasser (DE-588)1280300140 aut Géron, Aurélien (DE-588)1131560930 wpr Digitalisierung UB Bamberg - ADAM Catalogue Enrichment application/pdf http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=033692262&sequence=000001&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA Inhaltsverzeichnis |
| spellingShingle | Tunstall, Lewis Werra, Leandro von Wolf, Thomas Natural language processing with transformers building language applications with hugging face Deep Learning (DE-588)1135597375 gnd Natürliche Sprache (DE-588)4041354-8 gnd Textverarbeitung (DE-588)4059667-9 gnd Sprachverarbeitung (DE-588)4116579-2 gnd Maschinelles Lernen (DE-588)4193754-5 gnd |
| subject_GND | (DE-588)1135597375 (DE-588)4041354-8 (DE-588)4059667-9 (DE-588)4116579-2 (DE-588)4193754-5 |
| title | Natural language processing with transformers building language applications with hugging face |
| title_auth | Natural language processing with transformers building language applications with hugging face |
| title_exact_search | Natural language processing with transformers building language applications with hugging face |
| title_exact_search_txtP | Natural language processing with transformers building language applications with hugging face |
| title_full | Natural language processing with transformers building language applications with hugging face Lewis Tunstall, Leandro von Werra, and Thomas Wolf ; foreword by Aurélien Géron |
| title_fullStr | Natural language processing with transformers building language applications with hugging face Lewis Tunstall, Leandro von Werra, and Thomas Wolf ; foreword by Aurélien Géron |
| title_full_unstemmed | Natural language processing with transformers building language applications with hugging face Lewis Tunstall, Leandro von Werra, and Thomas Wolf ; foreword by Aurélien Géron |
| title_short | Natural language processing with transformers |
| title_sort | natural language processing with transformers building language applications with hugging face |
| title_sub | building language applications with hugging face |
| topic | Deep Learning (DE-588)1135597375 gnd Natürliche Sprache (DE-588)4041354-8 gnd Textverarbeitung (DE-588)4059667-9 gnd Sprachverarbeitung (DE-588)4116579-2 gnd Maschinelles Lernen (DE-588)4193754-5 gnd |
| topic_facet | Deep Learning Natürliche Sprache Textverarbeitung Sprachverarbeitung Maschinelles Lernen |
| url | http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=033692262&sequence=000001&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA |
| work_keys_str_mv | AT tunstalllewis naturallanguageprocessingwithtransformersbuildinglanguageapplicationswithhuggingface AT werraleandrovon naturallanguageprocessingwithtransformersbuildinglanguageapplicationswithhuggingface AT wolfthomas naturallanguageprocessingwithtransformersbuildinglanguageapplicationswithhuggingface AT geronaurelien naturallanguageprocessingwithtransformersbuildinglanguageapplicationswithhuggingface |