Neural network supervision: notes on loss functions, labels and confidence estimation:
Gespeichert in:
| 1. Verfasser: | |
|---|---|
| Format: | Abschlussarbeit Buch |
| Sprache: | English |
| Veröffentlicht: |
Passau
2019
|
| Schlagworte: | |
| Online-Zugang: | Volltext Volltext Inhaltsverzeichnis |
| Beschreibung: | ix, 98 Seiten Illustrationen, Diagramme |
Internformat
MARC
| LEADER | 00000nam a2200000 c 4500 | ||
|---|---|---|---|
| 001 | BV046829394 | ||
| 003 | DE-604 | ||
| 005 | 20200911 | ||
| 007 | t | ||
| 008 | 200729s2019 a||| m||| 00||| eng d | ||
| 035 | |a (OCoLC)1193292631 | ||
| 035 | |a (DE-599)BVBBV046829394 | ||
| 040 | |a DE-604 |b ger |e rda | ||
| 041 | 0 | |a eng | |
| 049 | |a DE-384 |a DE-473 |a DE-703 |a DE-1051 |a DE-824 |a DE-29 |a DE-12 |a DE-91 |a DE-19 |a DE-1049 |a DE-92 |a DE-739 |a DE-898 |a DE-355 |a DE-706 |a DE-20 |a DE-1102 |a DE-860 |a DE-2174 | ||
| 084 | |a ST 301 |0 (DE-625)143651: |2 rvk | ||
| 100 | 1 | |a Keren, Gil |d ca. 20./21. Jhd. |e Verfasser |0 (DE-588)121750219X |4 aut | |
| 245 | 1 | 0 | |a Neural network supervision: notes on loss functions, labels and confidence estimation |c Gil Keren |
| 264 | 1 | |a Passau |c 2019 | |
| 300 | |a ix, 98 Seiten |b Illustrationen, Diagramme | ||
| 336 | |b txt |2 rdacontent | ||
| 337 | |b n |2 rdamedia | ||
| 338 | |b nc |2 rdacarrier | ||
| 502 | |b Dissertation |c Universität Passau |d 2020 | ||
| 650 | 0 | 7 | |a Maschinelles Lernen |0 (DE-588)4193754-5 |2 gnd |9 rswk-swf |
| 650 | 0 | 7 | |a Neuronales Netz |0 (DE-588)4226127-2 |2 gnd |9 rswk-swf |
| 655 | 7 | |0 (DE-588)4113937-9 |a Hochschulschrift |2 gnd-content | |
| 689 | 0 | 0 | |a Neuronales Netz |0 (DE-588)4226127-2 |D s |
| 689 | 0 | 1 | |a Maschinelles Lernen |0 (DE-588)4193754-5 |D s |
| 689 | 0 | |5 DE-604 | |
| 776 | 0 | 8 | |i Erscheint auch als |n Online-Ausgabe |o urn:nbn:de:bvb:739-opus4-8223 |
| 856 | 4 | 1 | |u https://opus4.kobv.de/opus4-uni-passau/frontdoor/index/index/docId/822 |z kostenfrei |3 Volltext |
| 856 | 4 | 1 | |u https://nbn-resolving.de/urn:nbn:de:bvb:739-opus4-8223 |x Resolving-System |z kostenfrei |3 Volltext |
| 856 | 4 | 2 | |m Digitalisierung UB Passau - ADAM Catalogue Enrichment |q application/pdf |u http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=032238592&sequence=000001&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA |3 Inhaltsverzeichnis |
| 912 | |a ebook | ||
| 999 | |a oai:aleph.bib-bvb.de:BVB01-032238592 | ||
Datensatz im Suchindex
| _version_ | 1804181649179541504 |
|---|---|
| adam_text | Table of contents Nomenclature ix 1 Introduction 1.1 Training without a loss function .................................................................. 1.2 Differences between loss functions............................................................... 1.3 Low-quality supervision ............................................................................... 1.4 Calibrated prediction intervals...................................................................... 1 1 3 4 6 2 Tbnable Sensitivityfor Large Errors 2.1 Motivation..................................................................................................... 2.2 Related work............................................................................................... 2.3 Linear dependence on classification error ................................................... 2.4 Generalising the gradient.............................................................................. 2.5 Constraints on the pseudo-gradient............................................................... 2.6 Polynomial dependence on the classification error....................................... 2.7 Non-existence of a loss function.................................................................. 2.8 A toy example................................................................................................ 9 9 10 11 12 13 15 15 18 2.9 Experiments.................................................................................................. 21 The Principle of Logit Separation 3.1
Motivation...................................................................................................... 3.2 Related work................................................................................................ 3.3 The Principle of Logit Separation.................................................................. 27 27 30 31 3.4 32 32 33 33 35 3 Existing objectives that do not satisfy the PoLS.......................................... 3.4.1 The cross-entropy loss..................................................................... 3.4.2 The max-margin loss........................................................................ 3.4.3 Softmax Cauchy-Schwarz divergence............................................. 3.4.4 Sigmoid Cauchy-Schwarz divergence.............................................
viii Table of contents 3.5 3.6 3.7 4 35 36 36 38 39 39 40 40 42 43 43 46 47 Weakly Supervised One-Shot Detection 51 4.1 4.2 4.3 51 53 54 55 56 58 59 59 59 61 62 63 4.4 4.5 5 3.4.5 Softmax Tanimoto loss...................................................................... Existing objectives that satisfy the PoLS...................................................... 3.5.1 Self-normalisation........................................................................... 3.5.2 Noise Contrastive Estimation............................................................ 3.5.3 Binary cross-entropy........................................................................ 3.5.4 Sigmoid Tanimoto loss..................................................................... Novel objectives that satisfy the PoLS......................................................... 3.6.1 Batch cross-entropy ........................................................................ 3.6.2 Batch max-margin........................................................................... Experiments................................................................................................... 3.7.1 PoLS and SLC accuracy.................................................................. 3.7.2 SLC vs computing all logits............................................................ 3.7.3 SLC speedups ................................................................................. Motivation...................................................................................................... Related
work................................................................................................ Method......................................................................................................... 4.3.1 Similarity scores.............................................................................. 4.3.2 Weakly supervised detection............................................................ 4.3.3 One-shot learning.............................................................................. 4.3.4 Detection.......................................................................................... Experiments................................................................................................... Audio data...................................................................................................... 4.5.1 Computer vision data........................................................................ 4.5.2 Network specifications..................................................................... 4.5.3 Evaluation ....................................................................................... Calibrated Prediction Intervals 67 5.1 5.2 5.3 5.4 67 69 70 71 73 74 75 75 5.5 Motivation...................................................................................................... Related work................................................................................................ Posterior prediction intervals........................................................................ Calibrated prediction
intervals..................................................................... 5.4.1 Empirical calibration........................................................................ 5.4.2 Temperature scaling........................................................................ Experiments................................................................................................... 5.5.1 Age prediction (audio)......................................................................
Table of contents 5.5.2 5.5.3 5.5.4 5.5.5 5.5.6 5.5.7 6 ix SNR prediction................................................................................ Age prediction (images)................................................................. ISO speed prediction....................................................................... Neural networks............................................................................. Calibration results .......................................................................... Regression results .......................................................................... 76 76 77 77 78 80 Conclusion 83 6.1 6.2 6.3 6.4 83 84 86 87 Training without a loss function ................................................................. Differences between loss functions............................................................... Low-quality supervision ............................................................................. Calibrated prediction intervals.................................................................... References 89
|
| adam_txt |
Table of contents Nomenclature ix 1 Introduction 1.1 Training without a loss function . 1.2 Differences between loss functions. 1.3 Low-quality supervision . 1.4 Calibrated prediction intervals. 1 1 3 4 6 2 Tbnable Sensitivityfor Large Errors 2.1 Motivation. 2.2 Related work. 2.3 Linear dependence on classification error . 2.4 Generalising the gradient. 2.5 Constraints on the pseudo-gradient. 2.6 Polynomial dependence on the classification error. 2.7 Non-existence of a loss function. 2.8 A toy example. 9 9 10 11 12 13 15 15 18 2.9 Experiments. 21 The Principle of Logit Separation 3.1
Motivation. 3.2 Related work. 3.3 The Principle of Logit Separation. 27 27 30 31 3.4 32 32 33 33 35 3 Existing objectives that do not satisfy the PoLS. 3.4.1 The cross-entropy loss. 3.4.2 The max-margin loss. 3.4.3 Softmax Cauchy-Schwarz divergence. 3.4.4 Sigmoid Cauchy-Schwarz divergence.
viii Table of contents 3.5 3.6 3.7 4 35 36 36 38 39 39 40 40 42 43 43 46 47 Weakly Supervised One-Shot Detection 51 4.1 4.2 4.3 51 53 54 55 56 58 59 59 59 61 62 63 4.4 4.5 5 3.4.5 Softmax Tanimoto loss. Existing objectives that satisfy the PoLS. 3.5.1 Self-normalisation. 3.5.2 Noise Contrastive Estimation. 3.5.3 Binary cross-entropy. 3.5.4 Sigmoid Tanimoto loss. Novel objectives that satisfy the PoLS. 3.6.1 Batch cross-entropy . 3.6.2 Batch max-margin. Experiments. 3.7.1 PoLS and SLC accuracy. 3.7.2 SLC vs computing all logits. 3.7.3 SLC speedups . Motivation. Related
work. Method. 4.3.1 Similarity scores. 4.3.2 Weakly supervised detection. 4.3.3 One-shot learning. 4.3.4 Detection. Experiments. Audio data. 4.5.1 Computer vision data. 4.5.2 Network specifications. 4.5.3 Evaluation . Calibrated Prediction Intervals 67 5.1 5.2 5.3 5.4 67 69 70 71 73 74 75 75 5.5 Motivation. Related work. Posterior prediction intervals. Calibrated prediction
intervals. 5.4.1 Empirical calibration. 5.4.2 Temperature scaling. Experiments. 5.5.1 Age prediction (audio).
Table of contents 5.5.2 5.5.3 5.5.4 5.5.5 5.5.6 5.5.7 6 ix SNR prediction. Age prediction (images). ISO speed prediction. Neural networks. Calibration results . Regression results . 76 76 77 77 78 80 Conclusion 83 6.1 6.2 6.3 6.4 83 84 86 87 Training without a loss function . Differences between loss functions. Low-quality supervision . Calibrated prediction intervals. References 89 |
| any_adam_object | 1 |
| any_adam_object_boolean | 1 |
| author | Keren, Gil ca. 20./21. Jhd |
| author_GND | (DE-588)121750219X |
| author_facet | Keren, Gil ca. 20./21. Jhd |
| author_role | aut |
| author_sort | Keren, Gil ca. 20./21. Jhd |
| author_variant | g k gk |
| building | Verbundindex |
| bvnumber | BV046829394 |
| classification_rvk | ST 301 |
| collection | ebook |
| ctrlnum | (OCoLC)1193292631 (DE-599)BVBBV046829394 |
| discipline | Informatik |
| discipline_str_mv | Informatik |
| format | Thesis Book |
| fullrecord | <?xml version="1.0" encoding="UTF-8"?><collection xmlns="http://www.loc.gov/MARC21/slim"><record><leader>01921nam a2200397 c 4500</leader><controlfield tag="001">BV046829394</controlfield><controlfield tag="003">DE-604</controlfield><controlfield tag="005">20200911 </controlfield><controlfield tag="007">t</controlfield><controlfield tag="008">200729s2019 a||| m||| 00||| eng d</controlfield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(OCoLC)1193292631</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-599)BVBBV046829394</subfield></datafield><datafield tag="040" ind1=" " ind2=" "><subfield code="a">DE-604</subfield><subfield code="b">ger</subfield><subfield code="e">rda</subfield></datafield><datafield tag="041" ind1="0" ind2=" "><subfield code="a">eng</subfield></datafield><datafield tag="049" ind1=" " ind2=" "><subfield code="a">DE-384</subfield><subfield code="a">DE-473</subfield><subfield code="a">DE-703</subfield><subfield code="a">DE-1051</subfield><subfield code="a">DE-824</subfield><subfield code="a">DE-29</subfield><subfield code="a">DE-12</subfield><subfield code="a">DE-91</subfield><subfield code="a">DE-19</subfield><subfield code="a">DE-1049</subfield><subfield code="a">DE-92</subfield><subfield code="a">DE-739</subfield><subfield code="a">DE-898</subfield><subfield code="a">DE-355</subfield><subfield code="a">DE-706</subfield><subfield code="a">DE-20</subfield><subfield code="a">DE-1102</subfield><subfield code="a">DE-860</subfield><subfield code="a">DE-2174</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">ST 301</subfield><subfield code="0">(DE-625)143651:</subfield><subfield code="2">rvk</subfield></datafield><datafield tag="100" ind1="1" ind2=" "><subfield code="a">Keren, Gil</subfield><subfield code="d">ca. 20./21. Jhd.</subfield><subfield code="e">Verfasser</subfield><subfield code="0">(DE-588)121750219X</subfield><subfield code="4">aut</subfield></datafield><datafield tag="245" ind1="1" ind2="0"><subfield code="a">Neural network supervision: notes on loss functions, labels and confidence estimation</subfield><subfield code="c">Gil Keren</subfield></datafield><datafield tag="264" ind1=" " ind2="1"><subfield code="a">Passau</subfield><subfield code="c">2019</subfield></datafield><datafield tag="300" ind1=" " ind2=" "><subfield code="a">ix, 98 Seiten</subfield><subfield code="b">Illustrationen, Diagramme</subfield></datafield><datafield tag="336" ind1=" " ind2=" "><subfield code="b">txt</subfield><subfield code="2">rdacontent</subfield></datafield><datafield tag="337" ind1=" " ind2=" "><subfield code="b">n</subfield><subfield code="2">rdamedia</subfield></datafield><datafield tag="338" ind1=" " ind2=" "><subfield code="b">nc</subfield><subfield code="2">rdacarrier</subfield></datafield><datafield tag="502" ind1=" " ind2=" "><subfield code="b">Dissertation</subfield><subfield code="c">Universität Passau</subfield><subfield code="d">2020</subfield></datafield><datafield tag="650" ind1="0" ind2="7"><subfield code="a">Maschinelles Lernen</subfield><subfield code="0">(DE-588)4193754-5</subfield><subfield code="2">gnd</subfield><subfield code="9">rswk-swf</subfield></datafield><datafield tag="650" ind1="0" ind2="7"><subfield code="a">Neuronales Netz</subfield><subfield code="0">(DE-588)4226127-2</subfield><subfield code="2">gnd</subfield><subfield code="9">rswk-swf</subfield></datafield><datafield tag="655" ind1=" " ind2="7"><subfield code="0">(DE-588)4113937-9</subfield><subfield code="a">Hochschulschrift</subfield><subfield code="2">gnd-content</subfield></datafield><datafield tag="689" ind1="0" ind2="0"><subfield code="a">Neuronales Netz</subfield><subfield code="0">(DE-588)4226127-2</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="0" ind2="1"><subfield code="a">Maschinelles Lernen</subfield><subfield code="0">(DE-588)4193754-5</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="0" ind2=" "><subfield code="5">DE-604</subfield></datafield><datafield tag="776" ind1="0" ind2="8"><subfield code="i">Erscheint auch als</subfield><subfield code="n">Online-Ausgabe</subfield><subfield code="o">urn:nbn:de:bvb:739-opus4-8223</subfield></datafield><datafield tag="856" ind1="4" ind2="1"><subfield code="u">https://opus4.kobv.de/opus4-uni-passau/frontdoor/index/index/docId/822</subfield><subfield code="z">kostenfrei</subfield><subfield code="3">Volltext</subfield></datafield><datafield tag="856" ind1="4" ind2="1"><subfield code="u">https://nbn-resolving.de/urn:nbn:de:bvb:739-opus4-8223</subfield><subfield code="x">Resolving-System</subfield><subfield code="z">kostenfrei</subfield><subfield code="3">Volltext</subfield></datafield><datafield tag="856" ind1="4" ind2="2"><subfield code="m">Digitalisierung UB Passau - ADAM Catalogue Enrichment</subfield><subfield code="q">application/pdf</subfield><subfield code="u">http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=032238592&sequence=000001&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA</subfield><subfield code="3">Inhaltsverzeichnis</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">ebook</subfield></datafield><datafield tag="999" ind1=" " ind2=" "><subfield code="a">oai:aleph.bib-bvb.de:BVB01-032238592</subfield></datafield></record></collection> |
| genre | (DE-588)4113937-9 Hochschulschrift gnd-content |
| genre_facet | Hochschulschrift |
| id | DE-604.BV046829394 |
| illustrated | Illustrated |
| index_date | 2024-07-03T15:04:22Z |
| indexdate | 2024-07-10T08:55:00Z |
| institution | BVB |
| language | English |
| oai_aleph_id | oai:aleph.bib-bvb.de:BVB01-032238592 |
| oclc_num | 1193292631 |
| open_access_boolean | 1 |
| owner | DE-384 DE-473 DE-BY-UBG DE-703 DE-1051 DE-824 DE-29 DE-12 DE-91 DE-BY-TUM DE-19 DE-BY-UBM DE-1049 DE-92 DE-739 DE-898 DE-BY-UBR DE-355 DE-BY-UBR DE-706 DE-20 DE-1102 DE-860 DE-2174 |
| owner_facet | DE-384 DE-473 DE-BY-UBG DE-703 DE-1051 DE-824 DE-29 DE-12 DE-91 DE-BY-TUM DE-19 DE-BY-UBM DE-1049 DE-92 DE-739 DE-898 DE-BY-UBR DE-355 DE-BY-UBR DE-706 DE-20 DE-1102 DE-860 DE-2174 |
| physical | ix, 98 Seiten Illustrationen, Diagramme |
| psigel | ebook |
| publishDate | 2019 |
| publishDateSearch | 2019 |
| publishDateSort | 2019 |
| record_format | marc |
| spelling | Keren, Gil ca. 20./21. Jhd. Verfasser (DE-588)121750219X aut Neural network supervision: notes on loss functions, labels and confidence estimation Gil Keren Passau 2019 ix, 98 Seiten Illustrationen, Diagramme txt rdacontent n rdamedia nc rdacarrier Dissertation Universität Passau 2020 Maschinelles Lernen (DE-588)4193754-5 gnd rswk-swf Neuronales Netz (DE-588)4226127-2 gnd rswk-swf (DE-588)4113937-9 Hochschulschrift gnd-content Neuronales Netz (DE-588)4226127-2 s Maschinelles Lernen (DE-588)4193754-5 s DE-604 Erscheint auch als Online-Ausgabe urn:nbn:de:bvb:739-opus4-8223 https://opus4.kobv.de/opus4-uni-passau/frontdoor/index/index/docId/822 kostenfrei Volltext https://nbn-resolving.de/urn:nbn:de:bvb:739-opus4-8223 Resolving-System kostenfrei Volltext Digitalisierung UB Passau - ADAM Catalogue Enrichment application/pdf http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=032238592&sequence=000001&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA Inhaltsverzeichnis |
| spellingShingle | Keren, Gil ca. 20./21. Jhd Neural network supervision: notes on loss functions, labels and confidence estimation Maschinelles Lernen (DE-588)4193754-5 gnd Neuronales Netz (DE-588)4226127-2 gnd |
| subject_GND | (DE-588)4193754-5 (DE-588)4226127-2 (DE-588)4113937-9 |
| title | Neural network supervision: notes on loss functions, labels and confidence estimation |
| title_auth | Neural network supervision: notes on loss functions, labels and confidence estimation |
| title_exact_search | Neural network supervision: notes on loss functions, labels and confidence estimation |
| title_exact_search_txtP | Neural network supervision: notes on loss functions, labels and confidence estimation |
| title_full | Neural network supervision: notes on loss functions, labels and confidence estimation Gil Keren |
| title_fullStr | Neural network supervision: notes on loss functions, labels and confidence estimation Gil Keren |
| title_full_unstemmed | Neural network supervision: notes on loss functions, labels and confidence estimation Gil Keren |
| title_short | Neural network supervision: notes on loss functions, labels and confidence estimation |
| title_sort | neural network supervision notes on loss functions labels and confidence estimation |
| topic | Maschinelles Lernen (DE-588)4193754-5 gnd Neuronales Netz (DE-588)4226127-2 gnd |
| topic_facet | Maschinelles Lernen Neuronales Netz Hochschulschrift |
| url | https://opus4.kobv.de/opus4-uni-passau/frontdoor/index/index/docId/822 https://nbn-resolving.de/urn:nbn:de:bvb:739-opus4-8223 http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=032238592&sequence=000001&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA |
| work_keys_str_mv | AT kerengil neuralnetworksupervisionnotesonlossfunctionslabelsandconfidenceestimation |