The numerical solution of neural network training problems:
Abstract: "The training problem for feedforward neural networks is nonlinear parameter estimation that can be solved by a variety of optimization techniques. Much of the literature on neural networks has focused on backpropagation which is a variant of gradient descent. The training of neural n...
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Buch |
| Sprache: | English |
| Veröffentlicht: |
Urbana, Ill.
1991
|
| Schriftenreihe: | Center for Supercomputing Research and Development <Urbana, Ill.>: CSRD report
1089 |
| Schlagworte: | |
| Zusammenfassung: | Abstract: "The training problem for feedforward neural networks is nonlinear parameter estimation that can be solved by a variety of optimization techniques. Much of the literature on neural networks has focused on backpropagation which is a variant of gradient descent. The training of neural networks using backpropagation is known to be a slow process with more sophisticated techniques not always performing significantly better. In this paper, we show that feedforward neural networks can have ill-conditioned Hessians and that this ill-conditioning can be quite common The analysis and experimental results in this paper lead to the conclusion that many network training problems are ill-conditioned and may not be solved more efficiently by higher order optimization methods. While our analyses are for completely connected networks, they extend to networks with sparse connectivity as well. Our results suggest that neural networks can have considerable redundancy in parameterizing the function space in a neighborhood of a local minimum, independently of whether or not the solution has a small residual. |
Internformat
MARC
| LEADER | 00000nam a2200000 cb4500 | ||
|---|---|---|---|
| 001 | BV008979070 | ||
| 003 | DE-604 | ||
| 005 | 00000000000000.0 | ||
| 007 | t | ||
| 008 | 940206s1991 |||| 00||| eng d | ||
| 035 | |a (OCoLC)27130036 | ||
| 035 | |a (DE-599)BVBBV008979070 | ||
| 040 | |a DE-604 |b ger |e rakddb | ||
| 041 | 0 | |a eng | |
| 049 | |a DE-29T | ||
| 100 | 1 | |a Saarinen, S. |e Verfasser |4 aut | |
| 245 | 1 | 0 | |a The numerical solution of neural network training problems |c S. Saarinen, R. Bramley and G. Cybenko |
| 264 | 1 | |a Urbana, Ill. |c 1991 | |
| 336 | |b txt |2 rdacontent | ||
| 337 | |b n |2 rdamedia | ||
| 338 | |b nc |2 rdacarrier | ||
| 490 | 1 | |a Center for Supercomputing Research and Development <Urbana, Ill.>: CSRD report |v 1089 | |
| 520 | 3 | |a Abstract: "The training problem for feedforward neural networks is nonlinear parameter estimation that can be solved by a variety of optimization techniques. Much of the literature on neural networks has focused on backpropagation which is a variant of gradient descent. The training of neural networks using backpropagation is known to be a slow process with more sophisticated techniques not always performing significantly better. In this paper, we show that feedforward neural networks can have ill-conditioned Hessians and that this ill-conditioning can be quite common | |
| 520 | 3 | |a The analysis and experimental results in this paper lead to the conclusion that many network training problems are ill-conditioned and may not be solved more efficiently by higher order optimization methods. While our analyses are for completely connected networks, they extend to networks with sparse connectivity as well. Our results suggest that neural networks can have considerable redundancy in parameterizing the function space in a neighborhood of a local minimum, independently of whether or not the solution has a small residual. | |
| 650 | 4 | |a Neural networks (Computer science) | |
| 700 | 1 | |a Bramley, R. |e Verfasser |4 aut | |
| 700 | 1 | |a Cybenko, George |e Verfasser |4 aut | |
| 830 | 0 | |a Center for Supercomputing Research and Development <Urbana, Ill.>: CSRD report |v 1089 |w (DE-604)BV008930033 |9 1089 | |
| 999 | |a oai:aleph.bib-bvb.de:BVB01-005929830 | ||
Datensatz im Suchindex
| _version_ | 1804123317848768512 |
|---|---|
| any_adam_object | |
| author | Saarinen, S. Bramley, R. Cybenko, George |
| author_facet | Saarinen, S. Bramley, R. Cybenko, George |
| author_role | aut aut aut |
| author_sort | Saarinen, S. |
| author_variant | s s ss r b rb g c gc |
| building | Verbundindex |
| bvnumber | BV008979070 |
| ctrlnum | (OCoLC)27130036 (DE-599)BVBBV008979070 |
| format | Book |
| fullrecord | <?xml version="1.0" encoding="UTF-8"?><collection xmlns="http://www.loc.gov/MARC21/slim"><record><leader>02177nam a2200313 cb4500</leader><controlfield tag="001">BV008979070</controlfield><controlfield tag="003">DE-604</controlfield><controlfield tag="005">00000000000000.0</controlfield><controlfield tag="007">t</controlfield><controlfield tag="008">940206s1991 |||| 00||| eng d</controlfield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(OCoLC)27130036</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-599)BVBBV008979070</subfield></datafield><datafield tag="040" ind1=" " ind2=" "><subfield code="a">DE-604</subfield><subfield code="b">ger</subfield><subfield code="e">rakddb</subfield></datafield><datafield tag="041" ind1="0" ind2=" "><subfield code="a">eng</subfield></datafield><datafield tag="049" ind1=" " ind2=" "><subfield code="a">DE-29T</subfield></datafield><datafield tag="100" ind1="1" ind2=" "><subfield code="a">Saarinen, S.</subfield><subfield code="e">Verfasser</subfield><subfield code="4">aut</subfield></datafield><datafield tag="245" ind1="1" ind2="0"><subfield code="a">The numerical solution of neural network training problems</subfield><subfield code="c">S. Saarinen, R. Bramley and G. Cybenko</subfield></datafield><datafield tag="264" ind1=" " ind2="1"><subfield code="a">Urbana, Ill.</subfield><subfield code="c">1991</subfield></datafield><datafield tag="336" ind1=" " ind2=" "><subfield code="b">txt</subfield><subfield code="2">rdacontent</subfield></datafield><datafield tag="337" ind1=" " ind2=" "><subfield code="b">n</subfield><subfield code="2">rdamedia</subfield></datafield><datafield tag="338" ind1=" " ind2=" "><subfield code="b">nc</subfield><subfield code="2">rdacarrier</subfield></datafield><datafield tag="490" ind1="1" ind2=" "><subfield code="a">Center for Supercomputing Research and Development <Urbana, Ill.>: CSRD report</subfield><subfield code="v">1089</subfield></datafield><datafield tag="520" ind1="3" ind2=" "><subfield code="a">Abstract: "The training problem for feedforward neural networks is nonlinear parameter estimation that can be solved by a variety of optimization techniques. Much of the literature on neural networks has focused on backpropagation which is a variant of gradient descent. The training of neural networks using backpropagation is known to be a slow process with more sophisticated techniques not always performing significantly better. In this paper, we show that feedforward neural networks can have ill-conditioned Hessians and that this ill-conditioning can be quite common</subfield></datafield><datafield tag="520" ind1="3" ind2=" "><subfield code="a">The analysis and experimental results in this paper lead to the conclusion that many network training problems are ill-conditioned and may not be solved more efficiently by higher order optimization methods. While our analyses are for completely connected networks, they extend to networks with sparse connectivity as well. Our results suggest that neural networks can have considerable redundancy in parameterizing the function space in a neighborhood of a local minimum, independently of whether or not the solution has a small residual.</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Neural networks (Computer science)</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">Bramley, R.</subfield><subfield code="e">Verfasser</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">Cybenko, George</subfield><subfield code="e">Verfasser</subfield><subfield code="4">aut</subfield></datafield><datafield tag="830" ind1=" " ind2="0"><subfield code="a">Center for Supercomputing Research and Development <Urbana, Ill.>: CSRD report</subfield><subfield code="v">1089</subfield><subfield code="w">(DE-604)BV008930033</subfield><subfield code="9">1089</subfield></datafield><datafield tag="999" ind1=" " ind2=" "><subfield code="a">oai:aleph.bib-bvb.de:BVB01-005929830</subfield></datafield></record></collection> |
| id | DE-604.BV008979070 |
| illustrated | Not Illustrated |
| indexdate | 2024-07-09T17:27:51Z |
| institution | BVB |
| language | English |
| oai_aleph_id | oai:aleph.bib-bvb.de:BVB01-005929830 |
| oclc_num | 27130036 |
| open_access_boolean | |
| owner | DE-29T |
| owner_facet | DE-29T |
| publishDate | 1991 |
| publishDateSearch | 1991 |
| publishDateSort | 1991 |
| record_format | marc |
| series | Center for Supercomputing Research and Development <Urbana, Ill.>: CSRD report |
| series2 | Center for Supercomputing Research and Development <Urbana, Ill.>: CSRD report |
| spelling | Saarinen, S. Verfasser aut The numerical solution of neural network training problems S. Saarinen, R. Bramley and G. Cybenko Urbana, Ill. 1991 txt rdacontent n rdamedia nc rdacarrier Center for Supercomputing Research and Development <Urbana, Ill.>: CSRD report 1089 Abstract: "The training problem for feedforward neural networks is nonlinear parameter estimation that can be solved by a variety of optimization techniques. Much of the literature on neural networks has focused on backpropagation which is a variant of gradient descent. The training of neural networks using backpropagation is known to be a slow process with more sophisticated techniques not always performing significantly better. In this paper, we show that feedforward neural networks can have ill-conditioned Hessians and that this ill-conditioning can be quite common The analysis and experimental results in this paper lead to the conclusion that many network training problems are ill-conditioned and may not be solved more efficiently by higher order optimization methods. While our analyses are for completely connected networks, they extend to networks with sparse connectivity as well. Our results suggest that neural networks can have considerable redundancy in parameterizing the function space in a neighborhood of a local minimum, independently of whether or not the solution has a small residual. Neural networks (Computer science) Bramley, R. Verfasser aut Cybenko, George Verfasser aut Center for Supercomputing Research and Development <Urbana, Ill.>: CSRD report 1089 (DE-604)BV008930033 1089 |
| spellingShingle | Saarinen, S. Bramley, R. Cybenko, George The numerical solution of neural network training problems Center for Supercomputing Research and Development <Urbana, Ill.>: CSRD report Neural networks (Computer science) |
| title | The numerical solution of neural network training problems |
| title_auth | The numerical solution of neural network training problems |
| title_exact_search | The numerical solution of neural network training problems |
| title_full | The numerical solution of neural network training problems S. Saarinen, R. Bramley and G. Cybenko |
| title_fullStr | The numerical solution of neural network training problems S. Saarinen, R. Bramley and G. Cybenko |
| title_full_unstemmed | The numerical solution of neural network training problems S. Saarinen, R. Bramley and G. Cybenko |
| title_short | The numerical solution of neural network training problems |
| title_sort | the numerical solution of neural network training problems |
| topic | Neural networks (Computer science) |
| topic_facet | Neural networks (Computer science) |
| volume_link | (DE-604)BV008930033 |
| work_keys_str_mv | AT saarinens thenumericalsolutionofneuralnetworktrainingproblems AT bramleyr thenumericalsolutionofneuralnetworktrainingproblems AT cybenkogeorge thenumericalsolutionofneuralnetworktrainingproblems |