Malware data science: attack detection and attribution
Gespeichert in:
| 1. Verfasser: | |
|---|---|
| Format: | Buch |
| Sprache: | English |
| Veröffentlicht: |
San Francisco, CA
No Starch Press
[2018]
|
| Schlagworte: | |
| Online-Zugang: | Inhaltsverzeichnis |
| Beschreibung: | Includes index |
| Beschreibung: | xxvi, 243 Seiten Illustrationen, Diagramme |
| ISBN: | 9781593278595 1593278594 |
Internformat
MARC
| LEADER | 00000nam a2200000 c 4500 | ||
|---|---|---|---|
| 001 | BV045233371 | ||
| 003 | DE-604 | ||
| 005 | 20190620 | ||
| 007 | t | ||
| 008 | 181016s2018 a||| |||| 00||| eng d | ||
| 020 | |a 9781593278595 |9 978-1-59327-859-5 | ||
| 020 | |a 1593278594 |9 1-59327-859-4 | ||
| 035 | |a (OCoLC)1059557137 | ||
| 035 | |a (DE-599)BVBBV045233371 | ||
| 040 | |a DE-604 |b ger |e rda | ||
| 041 | 0 | |a eng | |
| 049 | |a DE-945 |a DE-573 |a DE-11 |a DE-898 |a DE-739 | ||
| 084 | |a ST 276 |0 (DE-625)143642: |2 rvk | ||
| 084 | |a ST 277 |0 (DE-625)143643: |2 rvk | ||
| 084 | |a ST 530 |0 (DE-625)143679: |2 rvk | ||
| 100 | 1 | |a Saxe, Joshua |e Verfasser |0 (DE-588)1170595693 |4 aut | |
| 245 | 1 | 0 | |a Malware data science |b attack detection and attribution |c by Joshua Saxe with Hillary Sanders |
| 264 | 1 | |a San Francisco, CA |b No Starch Press |c [2018] | |
| 264 | 4 | |c © 2018 | |
| 300 | |a xxvi, 243 Seiten |b Illustrationen, Diagramme | ||
| 336 | |b txt |2 rdacontent | ||
| 337 | |b n |2 rdamedia | ||
| 338 | |b nc |2 rdacarrier | ||
| 500 | |a Includes index | ||
| 650 | 0 | 7 | |a Big Data |0 (DE-588)4802620-7 |2 gnd |9 rswk-swf |
| 650 | 0 | 7 | |a Computervirus |0 (DE-588)4214774-8 |2 gnd |9 rswk-swf |
| 650 | 0 | 7 | |a Computersicherheit |0 (DE-588)4274324-2 |2 gnd |9 rswk-swf |
| 689 | 0 | 0 | |a Big Data |0 (DE-588)4802620-7 |D s |
| 689 | 0 | 1 | |a Computervirus |0 (DE-588)4214774-8 |D s |
| 689 | 0 | 2 | |a Computersicherheit |0 (DE-588)4274324-2 |D s |
| 689 | 0 | |5 DE-604 | |
| 700 | 1 | |a Sanders, Hillary |e Sonstige |0 (DE-588)1170595758 |4 oth | |
| 856 | 4 | 2 | |m Digitalisierung UB Passau - ADAM Catalogue Enrichment |q application/pdf |u http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=030621711&sequence=000001&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA |3 Inhaltsverzeichnis |
| 999 | |a oai:aleph.bib-bvb.de:BVB01-030621711 | ||
Datensatz im Suchindex
| _version_ | 1804178962288476160 |
|---|---|
| adam_text | CONTENTS IN DETAIL FOREWORD by Anup Ghosh xvii ACKNOWLEDGMENTS xix INTRODUCTION xxi What Is Data Science?............................................................................................................ xxii Why Data Science Matters for Security.................................................................................. xxii Applying Data Science to Malware......................................................................................xxiii Who Should Read This Book?................................................................................................xxiv About This Book....................................................................................................................... xxiv How to Use the Sample Code and Data................................................................................xxv 1 BASIC STATIC MALWARE ANALYSIS 1 The Microsoft Windows Portable Executable Format................................................................ 2 The PE Header........................................................................................................... 3 The Optional Header............................................................................................... 3 Section Headers........................................................................................................ 4 Dissecting the PE Format Using pefile........................................................................................ 5 Examining Malware
Images...................................................................................................... 7 Examining Malware Strings.........................................................................................................8 Using the strings Program........................................................................................ 8 Analyzing Your strings Dump................................................................................... 9 Summary................................................................................................................................ 10 2 BEYOND BASIC STATIC ANALYSIS: X86 DISASSEMBLY 11 Disassembly Methods............................................................................................................. 12 Basics of x86 Assembly Language........................................................................................ 12 CPU Registers........................................................................................................ 13 Arithmetic Instructions.......................................................................................... 15 Data Movement Instructions................................................................................. 15 Disassembling ircbot.exe Using pefile and capstone.............................................................. 20 Factors That Limit Static Analysis.............................................................................................21
Packing.................................................................................................................... 21 Resource Obfuscation.............................................................................................22 Anti-disassembly Techniques................................................................................... 22 Dynamically Downloaded Data............................................................................ 22 Summary...................................................................................................................................23
3 A BRIEF INTRODUCTION ТО DYNAMIC ANALYSIS 6 25 Why Use Dynamic Analysis?........................................................................................................ 26 Dynamic Analysis for Malware Data Science............................................................................. 26 Basic Tools for Dynamic Analysis.................................................................................................27 Typical Malware Behaviors....................................................................................... 27 Loading a File on malwr.com.....................................................................................27 Analyzing Results on malwr.com................................................................................28 Limitations of Basic Dynamic Analysis......................................................................................... 33 Summary........................................................................................................................................ 34 4 IDENTIFYING ATTACK CAMPAIGNS USING MALWARE NETWORKS 35 Nodes and Edges..........................................................................................................................37 Bipartite Networks..........................................................................................................................37 Visualizing Malware Networks................................................................................................... 39 The Distortion
Problem.................................................................................................39 Force-Directed Algorithms......................................................................................... 40 Building Networks with NetworkX.............................................................................................. 40 Adding Nodes and Edges............................................................................................................. 41 Adding Attributes........................................................................................................ 42 Saving Networks to Disk............................................................................................42 Network Visualization with GraphViz......................................................................................... 43 Using Parameters to Adjust Networks..................................................................... 44 The GraphViz Command Line Tools........................................................................44 Adding Visual Attributes to Nodes and Edges.......................................................48 Building Malware Networks........................................................................................................ 51 Building a Shared Image Relationship Network........................................................................ 54 Summary........................................................................................................................................ 58 5 SHARED CODE ANALYSIS 59
Preparing Samples for Comparison by Extracting Features.................................................... 62 How Bag of Features Models Work.......................................................................... 62 What are N-Grams?................................................................................................... 63 Using the Jaccard Index to Quantify Similarity.......................................................................... 64 Using Similarity Matrices to Evaluate Malware Shared Code Estimation Methods.............66 Instruction Sequence-Based Similarity..................................................................... 67 Strings-Based Similarity.............................................................................................. 70 Import Address Table-Based Similarity................................................................... 71 Dynamic API Call-Based Similarity...........................................................................72 Building a Similarity Graph...........................................................................................................73 Scaling Similarity Comparisons................................................................................................... 77 Minhash in a Nutshell.................................................................................................77 Minhash in Depth........................................................................................................ 78 Building a Persistent Malware Similarity Search
System......................................................... 79 Running the Similarity Search System..........................................................................................85 Summary.........................................................................................................................................87 XII Contents in Detail UNDERSTANDING MACHINE LEARNING-BASED MALWARE DETECTORS 89 Steps for Building a Machine Learning-Based Detector............................................................90 Gathering Training Examples.....................................................................................91 Extracting Features...................................................................................................... 91 Designing Good Features............................................................................................ 92 Training Machine Learning Systems...........................................................................92 Testing Machine Learning Systems...........................................................................93 Understanding Feature Spaces and Decision Boundaries....................................................... 93 What Makes Models Good or Bad: Overfitting and Underfitting........................................98 Major Types of Machine Learning Algorithms..................................................................... 101 Logistic Regression................................................................................................. 102 К-Nearest
Neighbors.............................................................................................. 105 Decision Trees........................................................................................................ 109 Random Forest........................................................................................................ 115 Summary................................................................................................................................... ļ ļ 7 7 EVALUATING MALWARE DETECTION SYSTEMS 119 Four Possible Detection Outcomes......................................................................................... True and False Positive Rates............................................................................... Relationship Between True and False Positive Rates.......................................... ROC Curves............................................................................................................... Considering Base Rates in Your Evaluation............................................................................ How Base Rate Affects Precision............................................................................ Estimating Precision in a Deployment Environment......................................... Summary................................................................................................................................... 120 120 121 123 124 124 125 ļ շ ք 8 BUILDING MACHINE LEARNING DETECTORS 127 Terminology and
Concepts....................................................................................................... Building a Toy Decision Tree-Based Detector........................................................................ Training Your Decision Tree Classifier.................................................................. Visualizing the Decision Tree............................................................................... Complete Sample Code......................................................................................... Building Real-World Machine Learning Detectors with sklearn.......................................... Real-World Feature Extraction............................................................................. Why You Can t Use All Possible Features........................................................... Using the Hashing Trick to Compress Features.................................................. Building an Industrial-Strength Detector.................................................................................. Extracting Features................................................................................................. Training the Detector.............................................................................................. Running the Detector on New Binaries................................................................ What We ve Implemented So Far........................................................................ Evaluating Your Detector s
Performance................................................................................. Using ROC Curves to Evaluate Detector Efficacy............................................... Computing ROC Curves......................................................................................... Splitting Data into Training and Test Sets........................................................... 128 129 130 131 133 134 134 137 138 141 141 142 144 144 146 147 147 148 Contents in Detali ХІІІ
Computing the ROC Curve.................................................................................... Cross-Validation...................................................................................................... Next Steps................................................................................................................................. Summary................................................................................................................................... 9 VISUALIZING MALWARE TRENDS 149 150 153 154 155 Why Visualizing Malware Data Is Important........................................................................... 156 Understanding Our Malware Dataset.................................................................................... 158 Loading Data into pandas.................................................................................... 158 Working with a pandas DataFrame................................................................... 159 Filtering Data Using Conditions.......................................................................... 161 Using matplotlib to Visualize Data......................................................................................... 162 Plotting the Relationship Between Malware Size and Detection...................... 162 Plotting Ransomware Detection Rates................................................................... 164 Plotting Ransomware and Worm Detection Rates............................................. 165 Using seaborn to Visualize
Data............................................................................................ 168 Plotting the Distribution of Antivirus Detections.................................................. 169 Creating a Violin Plot............................................................................................ 172 Summary................................................................................................................................... 174 10 DEEP LEARNING BASICS What Is Deep Learning?........................................................................................................... How Neural Networks Work................................................................................................... Anatomy of a Neuron............................................................................................ A Network of Neurons......................................................................................... Universal Approximation Theorem ..................................................................... Building Your Own Neural Network................................................................... Adding Another Neuron to the Network........................................................... Automatic Feature Generation............................................................................. Training Neural Networks........................................................................................................ Using Backpropagation to Optimize a Neural Network................................... Path
Explosion........................................................................................................ Vanishing Gradient................................................................................................. Types of Neural Networks........................................................................................................ Feed-Forward Neural Network............................................................................. Convolutional Neural Network............................................................................. Autoencoder Neural Network............................................................................... Generative Adversarial Network.......................................................................... Recurrent Neural Network.................................................................................... ResNet..................................................................................................................... Summary................................................................................................................................... 175 176 177 177 180 181 182 186 188 189 190 192 192 193 193 193 194 195 196 196 197 12 BECOMING A DATA SCIENTIST 215 Paths to Becoming a Security Data Scientist.......................................................................... 216 A Day in the Life of a Security Data Scientist........................................................................216 Traits of an Effective Security Data
Scientist.......................................................................... 218 Open-Mindedness...................................................................................................... 218 Boundless Curiosity................................................................................................... 218 Obsession with Results...............................................................................................219 Skepticism of Results................................................................................................. 219 Where to Go from Here............................................................................................................. 219 APPENDIX AN OVERVIEW OF DATASETS AND TOOLS 221 Overview of Datasets.................................................................................................................. շշշ Chapter 1 : Basic Static Malware Analysis......................................................... 222 Chapter 2: Beyond Basic Static Analysis: x86 Disassembly........................... 222 Chapter 3: A Brief Introduction to Dynamic Analysis........................................222 Chapter 4: Identifying Attack Campaigns Using Malware Networks............ 222 Chapter 5: Shared Code Analysis...........................................................................223 Chapter 6: Understanding Machine Learning-Based Malware Detectors and Chapter 7: Evaluating Malware Detection Systems.................................223 Chapter 8: Building Machine Learning
Detectors................................................224 Chapter 9: Visualizing Malware Trends.................................................................224 Chapter 10: Deep Learning Basics........................................................................ 224 Chapter 11 : Building a Neural Network Malware Detector with Keras .... 224 Chapter 12: Becoming a Data Scientist.................................................................224 Tool Implementation Guide........................................................................................................ 225 Shared Hostname Network Visualization.............................................................. 225 Shared Image Network Visualization......................................................................226 Malware Similarity Visualization............................................................................. 227 Malware Similarity Search System........................................................................ 229 Machine Learning Malware Detection System.......................................................230 INDEX 11 BUILDING A NEURAL NETWORK MALWARE DETECTOR WITH KERAS Training the Model....................................................................................................................... 203 Extracting Features....................................................................................................203 Creating a Data Generator.....................................................................................204 Incorporating Validation
Data................................................................................207 Saving and Loading the Model................................................................................209 Evaluating the Model.................................................................................................................. 209 Enhancing the Model Training Process with Callbacks......................................................... 211 Using a Built-in Callback..........................................................................................212 Using a Custom Callback..........................................................................................213 Summary...................................................................................................................................... 214 233 199 Defining a Model s Architecture................................................................................................. 200 Compiling the Model.................................................................................................................. 202 Contents in Detail
|
| any_adam_object | 1 |
| author | Saxe, Joshua |
| author_GND | (DE-588)1170595693 (DE-588)1170595758 |
| author_facet | Saxe, Joshua |
| author_role | aut |
| author_sort | Saxe, Joshua |
| author_variant | j s js |
| building | Verbundindex |
| bvnumber | BV045233371 |
| classification_rvk | ST 276 ST 277 ST 530 |
| ctrlnum | (OCoLC)1059557137 (DE-599)BVBBV045233371 |
| discipline | Informatik |
| format | Book |
| fullrecord | <?xml version="1.0" encoding="UTF-8"?><collection xmlns="http://www.loc.gov/MARC21/slim"><record><leader>01782nam a2200433 c 4500</leader><controlfield tag="001">BV045233371</controlfield><controlfield tag="003">DE-604</controlfield><controlfield tag="005">20190620 </controlfield><controlfield tag="007">t</controlfield><controlfield tag="008">181016s2018 a||| |||| 00||| eng d</controlfield><datafield tag="020" ind1=" " ind2=" "><subfield code="a">9781593278595</subfield><subfield code="9">978-1-59327-859-5</subfield></datafield><datafield tag="020" ind1=" " ind2=" "><subfield code="a">1593278594</subfield><subfield code="9">1-59327-859-4</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(OCoLC)1059557137</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-599)BVBBV045233371</subfield></datafield><datafield tag="040" ind1=" " ind2=" "><subfield code="a">DE-604</subfield><subfield code="b">ger</subfield><subfield code="e">rda</subfield></datafield><datafield tag="041" ind1="0" ind2=" "><subfield code="a">eng</subfield></datafield><datafield tag="049" ind1=" " ind2=" "><subfield code="a">DE-945</subfield><subfield code="a">DE-573</subfield><subfield code="a">DE-11</subfield><subfield code="a">DE-898</subfield><subfield code="a">DE-739</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">ST 276</subfield><subfield code="0">(DE-625)143642:</subfield><subfield code="2">rvk</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">ST 277</subfield><subfield code="0">(DE-625)143643:</subfield><subfield code="2">rvk</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">ST 530</subfield><subfield code="0">(DE-625)143679:</subfield><subfield code="2">rvk</subfield></datafield><datafield tag="100" ind1="1" ind2=" "><subfield code="a">Saxe, Joshua</subfield><subfield code="e">Verfasser</subfield><subfield code="0">(DE-588)1170595693</subfield><subfield code="4">aut</subfield></datafield><datafield tag="245" ind1="1" ind2="0"><subfield code="a">Malware data science</subfield><subfield code="b">attack detection and attribution</subfield><subfield code="c">by Joshua Saxe with Hillary Sanders</subfield></datafield><datafield tag="264" ind1=" " ind2="1"><subfield code="a">San Francisco, CA</subfield><subfield code="b">No Starch Press</subfield><subfield code="c">[2018]</subfield></datafield><datafield tag="264" ind1=" " ind2="4"><subfield code="c">© 2018</subfield></datafield><datafield tag="300" ind1=" " ind2=" "><subfield code="a">xxvi, 243 Seiten</subfield><subfield code="b">Illustrationen, Diagramme</subfield></datafield><datafield tag="336" ind1=" " ind2=" "><subfield code="b">txt</subfield><subfield code="2">rdacontent</subfield></datafield><datafield tag="337" ind1=" " ind2=" "><subfield code="b">n</subfield><subfield code="2">rdamedia</subfield></datafield><datafield tag="338" ind1=" " ind2=" "><subfield code="b">nc</subfield><subfield code="2">rdacarrier</subfield></datafield><datafield tag="500" ind1=" " ind2=" "><subfield code="a">Includes index</subfield></datafield><datafield tag="650" ind1="0" ind2="7"><subfield code="a">Big Data</subfield><subfield code="0">(DE-588)4802620-7</subfield><subfield code="2">gnd</subfield><subfield code="9">rswk-swf</subfield></datafield><datafield tag="650" ind1="0" ind2="7"><subfield code="a">Computervirus</subfield><subfield code="0">(DE-588)4214774-8</subfield><subfield code="2">gnd</subfield><subfield code="9">rswk-swf</subfield></datafield><datafield tag="650" ind1="0" ind2="7"><subfield code="a">Computersicherheit</subfield><subfield code="0">(DE-588)4274324-2</subfield><subfield code="2">gnd</subfield><subfield code="9">rswk-swf</subfield></datafield><datafield tag="689" ind1="0" ind2="0"><subfield code="a">Big Data</subfield><subfield code="0">(DE-588)4802620-7</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="0" ind2="1"><subfield code="a">Computervirus</subfield><subfield code="0">(DE-588)4214774-8</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="0" ind2="2"><subfield code="a">Computersicherheit</subfield><subfield code="0">(DE-588)4274324-2</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="0" ind2=" "><subfield code="5">DE-604</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">Sanders, Hillary</subfield><subfield code="e">Sonstige</subfield><subfield code="0">(DE-588)1170595758</subfield><subfield code="4">oth</subfield></datafield><datafield tag="856" ind1="4" ind2="2"><subfield code="m">Digitalisierung UB Passau - ADAM Catalogue Enrichment</subfield><subfield code="q">application/pdf</subfield><subfield code="u">http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=030621711&sequence=000001&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA</subfield><subfield code="3">Inhaltsverzeichnis</subfield></datafield><datafield tag="999" ind1=" " ind2=" "><subfield code="a">oai:aleph.bib-bvb.de:BVB01-030621711</subfield></datafield></record></collection> |
| id | DE-604.BV045233371 |
| illustrated | Illustrated |
| indexdate | 2024-07-10T08:12:18Z |
| institution | BVB |
| isbn | 9781593278595 1593278594 |
| language | English |
| oai_aleph_id | oai:aleph.bib-bvb.de:BVB01-030621711 |
| oclc_num | 1059557137 |
| open_access_boolean | |
| owner | DE-945 DE-573 DE-11 DE-898 DE-BY-UBR DE-739 |
| owner_facet | DE-945 DE-573 DE-11 DE-898 DE-BY-UBR DE-739 |
| physical | xxvi, 243 Seiten Illustrationen, Diagramme |
| publishDate | 2018 |
| publishDateSearch | 2018 |
| publishDateSort | 2018 |
| publisher | No Starch Press |
| record_format | marc |
| spelling | Saxe, Joshua Verfasser (DE-588)1170595693 aut Malware data science attack detection and attribution by Joshua Saxe with Hillary Sanders San Francisco, CA No Starch Press [2018] © 2018 xxvi, 243 Seiten Illustrationen, Diagramme txt rdacontent n rdamedia nc rdacarrier Includes index Big Data (DE-588)4802620-7 gnd rswk-swf Computervirus (DE-588)4214774-8 gnd rswk-swf Computersicherheit (DE-588)4274324-2 gnd rswk-swf Big Data (DE-588)4802620-7 s Computervirus (DE-588)4214774-8 s Computersicherheit (DE-588)4274324-2 s DE-604 Sanders, Hillary Sonstige (DE-588)1170595758 oth Digitalisierung UB Passau - ADAM Catalogue Enrichment application/pdf http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=030621711&sequence=000001&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA Inhaltsverzeichnis |
| spellingShingle | Saxe, Joshua Malware data science attack detection and attribution Big Data (DE-588)4802620-7 gnd Computervirus (DE-588)4214774-8 gnd Computersicherheit (DE-588)4274324-2 gnd |
| subject_GND | (DE-588)4802620-7 (DE-588)4214774-8 (DE-588)4274324-2 |
| title | Malware data science attack detection and attribution |
| title_auth | Malware data science attack detection and attribution |
| title_exact_search | Malware data science attack detection and attribution |
| title_full | Malware data science attack detection and attribution by Joshua Saxe with Hillary Sanders |
| title_fullStr | Malware data science attack detection and attribution by Joshua Saxe with Hillary Sanders |
| title_full_unstemmed | Malware data science attack detection and attribution by Joshua Saxe with Hillary Sanders |
| title_short | Malware data science |
| title_sort | malware data science attack detection and attribution |
| title_sub | attack detection and attribution |
| topic | Big Data (DE-588)4802620-7 gnd Computervirus (DE-588)4214774-8 gnd Computersicherheit (DE-588)4274324-2 gnd |
| topic_facet | Big Data Computervirus Computersicherheit |
| url | http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=030621711&sequence=000001&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA |
| work_keys_str_mv | AT saxejoshua malwaredatascienceattackdetectionandattribution AT sandershillary malwaredatascienceattackdetectionandattribution |