Application of hebbian and anti-hebbian learning to nanotechnology-based physical neural networks Number:7,412,428 from the United States Patent and Trademark Office (PTO) owispatent Methods and systems are disclosed herein in which a physical neural network can be configured utilizing nanotechnology. Such a physical neural network can comprise a plurality of molecular conductors (e.g., nanoconductors) which form neural connections between pre-synaptic and post-synaptic components of the physical neural network. Additionally, a learning mechanism can be applied for implementing Hebbian learning via the physical neural network. Such a learning mechanism can utilize a voltage gradient or voltage gradient dependencies to implement Hebbian and/or anti-Hebbian plasticity within the physical neural network. The learning mechanism can also utilize pre-synaptic and post-synaptic frequencies to provide Hebbian and/or anti-Hebbian learning within the physical neural network.<H nanotechnology, Application, Application, of, , 7412428.html, Patent, pto, 7412428

Title: Application of hebbian and anti-hebbian learning to nanotechnology-based physical neural networks

Abstract: Methods and systems are disclosed herein in which a physical neural network can be configured utilizing nanotechnology. Such a physical neural network can comprise a plurality of molecular conductors (e.g., nanoconductors) which form neural connections between pre-synaptic and post-synaptic components of the physical neural network. Additionally, a learning mechanism can be applied for implementing Hebbian learning via the physical neural network. Such a learning mechanism can utilize a voltage gradient or voltage gradient dependencies to implement Hebbian and/or anti-Hebbian plasticity within the physical neural network. The learning mechanism can also utilize pre-synaptic and post-synaptic frequencies to provide Hebbian and/or anti-Hebbian learning within the physical neural network.

Patent Number: 7,412,428 Issued on 08/12/2008 to Nugent

Inventors: Nugent; Alex (Santa Fe, NM)
Assignee: Knowmtech, LLC. (Albuquerque, NM)

Appl. No.: 10/748,631

Filed: December 30, 2003

Related U.S. Patent Documents


Application Number	Filing Date	Patent Number	Issue Date
10226191	Aug., 2002
10162524	Jun., 2002
10095273	Mar., 2002	6889216

Current U.S. Class: 706/26 ; 706/15; 706/25; 706/6; 977/700; 977/712; 977/742

Current International Class: G06E 1/00 (20060101)

Field of Search: 706/26,6,12,15,25 977/700,712,720,724,742

References Cited [Referenced By]

U.S. Patent Documents


2707223	April 1955	Hollman
3833894	September 1974	Aviram et al.
4802951	February 1989	Clark et al.
4926064	May 1990	Tapang
4974146	November 1990	Works et al.
4988891	January 1991	Mashiko
5315162	May 1994	McHardy et al.
5422983	June 1995	Castelaz et al.
5475794	December 1995	Mashiko
5589692	December 1996	Reed
5649063	July 1997	Bose
5670818	September 1997	Forouhi et al.
5706404	January 1998	Colak
5717832	February 1998	Steimle et al.
5761115	June 1998	Kozicki et al.
5783840	July 1998	Randall et al.
5812993	September 1998	Ginosar et al.
5896312	April 1999	Kozicki et al.
5904545	May 1999	Smith et al.
5914893	June 1999	Kozicki et al.
5951881	September 1999	Rogers et al.
5978782	November 1999	Neely
6026358	February 2000	Tomabechi
6084796	July 2000	Kozicki et al.
6128214	October 2000	Kuekes et al.
6245630	June 2001	Ishikawa
6248529	June 2001	Connolly
6256767	July 2001	Kuekes et al.
6282530	August 2001	Huang
6294450	September 2001	Chen et al.
6314019	November 2001	Kuekes et al.
6330553	December 2001	Uchikawa et al.
6335291	January 2002	Freeman
6339227	January 2002	Ellenbogen
6359288	March 2002	Ying et al.
6363369	March 2002	Liaw et al.
6383923	May 2002	Brown et al.
6389404	May 2002	Carson et al.
6407443	June 2002	Chen et al.
6418423	July 2002	Kambhatla et al.
6420092	July 2002	Yang et al.
6422450	July 2002	Zhou et al.
6423583	July 2002	Avouris et al.
6424961	July 2002	Ayala
6426134	July 2002	Lavin et al.
6445006	September 2002	Brandes et al.
6536106	March 2003	Jackson et al.
6620346	September 2003	Schultz et al.
6687686	February 2004	Nervegna et al.
6763340	July 2004	Burns et al.
6798692	September 2004	Kozicki et al.
6855329	February 2005	Shakesheff et al.
2001/0004471	June 2001	Zhang
2001/0023986	September 2001	Mancevski
2001/0024633	September 2001	Lee et al.
2001/0031900	October 2001	Margrave et al.
2001/0041160	November 2001	Margrave et al.
2001/0044114	November 2001	Connolly
2002/0001905	January 2002	Choi et al.
2002/0004028	January 2002	Margrave et al.
2002/0004136	January 2002	Gao et al.
2002/0030205	March 2002	Varshavsky
2002/0075126	June 2002	Reitz et al.
2002/0086124	July 2002	Margrave et al.
2002/0090468	July 2002	Goto et al.
2002/0102353	August 2002	Mauthner et al.
2003/0031438	February 2003	Kambe et al.
2003/0177450	September 2003	Nugent

Foreign Patent Documents


1 022 764	Jan., 2000	EP
1 046 613	Apr., 2000	EP
1 069 206	Jan., 2001	EP
1 100 106	May., 2001	EP
1 115 135	Jul., 2001	EP
1 134 304	Sep., 2001	EP
2071126	Jun., 1996	RU
WO 00/44094	Jul., 2000	WO
WO 03/017282	Feb., 2003	WO

Other References

"Solid-state thin-film memistor for electronic neural networks", S. Thakoor, A. Moopenn, T. Daud, and A. P. Thakoor, Journal of Applied Physics--Mar. 15, 1990--vol. 67, Issue 6, pp. 3132-3135. cited by examiner .
"Computational nanotechnology with carbon nanotubes and fullerenes", Srivastava, D. Menon, M. Kyeongjae Cho, NASA Ames Res. Center, Moffett Field, CA, Computing in Science & Engineering, Jul./Aug. 2001, vol. 3, Issue 4, pp. 42-55. cited by examiner .
Peter Weiss, "Circuitry in a Nanowire: Novel Growth Method May Transform Chips," Science News Online, vol. 161, No. 6; Feb. 9, 2002. cited by other .
Press Release, "Nanowire-based electronics and optics comes one step closer," Eureka Alert, American Chemical Society; Feb. 1, 2002. cited by other .
Weeks et al., "High-pressure nanolithography using low-energy electrons from a scanning tunneling microscope," Institute of Physics Publishing, Nanotechnology 13 (2002), pp. 38-42; Dec. 12, 2001. cited by other .
CMP Cientifica, "Nanotech: the tiny revolution"; CMP Cientifica, Nov. 2001. cited by other .
Diehl, et al., "Self-Assembled, Deterministic Carbon Nanotube Wiring Networks," Angew. Chem. Int. Ed. 2002, 41, No. 2; Received Oct. 22, 2001. cited by other .
G. Pirio, et al., "Fabrication and electrical characteristics of carbon nanotube field emission microcathodes with an integrated gate electrode," Institute of Physics Publishing, Nanotechnology 13 (2002), pp. 1-4, Oct. 2, 2001. cited by other .
Leslie Smith, "An Introduction to Neural Networks," Center for Cognitive and Computational Neuroscience, Dept. of Computing & Mathematics, University of Stirling, Oct. 25, 1996; http//www.cs.stir.ac.uk/.about.Iss/NNIntro/InvSlides.html. cited by other .
V. Derycke et al., "Carbon Nanotube Inter- and Intramolecular Logic Gates," American Chemical Society, Nano Letters, XXXX, vol. 0, No. 0, A-D. cited by other .
Mark K. Anderson, "Mega Steps Toward the Nanochip," Wired News, Apr. 27, 2001. cited by other .
Collins et al., "Engineering Carbon Nanotubes and Nanotube Circuits Using Electrical Breakdown," Science, vol. 292, pp. 706-709, Apr. 27, 2001. cited by other .
Landman et al., "Metal-Semiconductor Nanocontacts: Silicon Nanowires," Physical Review Letters, vol. 85, No. 9, Aug. 28, 2000. cited by other .
John G. Spooner, "Tiny tubes mean big chip advances," Cnet News.com, Tech News First, Apr. 26, 2001. cited by other .
Jeong-Mi Moon et al., "High-Yield Purification Process of Singlewalled Carbon Nanotubes," J. Phys. Chem. B 2001, 105, pp. 5677-5681. cited by other .
"A New Class of Nanostructure: Semiconducting Nanobelts Offer Potential for Nanosensors and Nanoelectronics," Mar. 12, 2001, http://www.sciencedaily.com/releases/2001/03/010309080953.htm. cited by other .
Hermanson et al., "Dielectrophoretic Assembly of Electrically Functional Microwires from Nanoparticle Suspensions," Materials Science, vol. 294, No. 5544, Issue of Nov. 2, 2001, pp. 1082-1086. cited by other .
Press Release, "Toshiba Demonstrates Operation of Single-Electron Transistor Circuit at Room Temperature," Toshiba, Jan. 10, 2001. cited by other .
J. Appenzeller et al., "Optimized contact configuration for the study of transport phenomena in ropes of single-wall carbon nanotubes," Applied Physics Letters, vol. 78, No. 21, pp. 3313-3315, May 21, 2001. cited by other .
David Rotman, "Molecular Memory, Replacing silicon with organic molecules could mean tiny supercomputers," Technology Review, May 2001, p. 46. cited by other .
Westervelt et al., "Molecular Electronics," NSF Functional Nanostructures Grant 9871810, NSF Partnership in Nanotechnology Conference, Jan. 29-30, 2001; http://www.unix.oit.umass.edu/.about.nano/NewFiles/FN19.sub.--Harva- rd.pdf. cited by other .
Niyogi et al., "Chromatographic Purification of Soluble Single-Walled Carbon Nanotubes (s-SWNTs)," J. Am. Chem. Soc 2001, 123, pp. 733-734. cited by other .
Duan et al., "Indium phosphide nanowires as building blocks for nanoscale electronic and optoelectronic devices," Nature, vol. 409, Jan. 4, 2001, pp. 67-69. cited by other .
Paulson, et al., "Tunable Resistance of a Carbon Nanotube-Graphite Interface," Science, vol. 290, Dec. 1, 2000, pp. 1742-1744. cited by other .
Wei et al., "Reliability and current carrying capacity of carbon nanotubes," Applied Physics Letters, vol. 79, No. 8, Aug. 20, 2001, pp. 1172-1174. cited by other .
Avouris et al., "Carbon nanotubes: nanomechanics, manipulation, and electronic devices," Applied Surface Science 141 (1999), pp. 201-209. cited by other .
Smith et al., "Electric-field assisted assembly and alignment of metallic nanowires," Applied Physics Letters, vol. 77, No. 9, Aug. 28, 2000, pp. 1399-1401. cited by other .
Hone et al., "Electrical and thermal transport properties of magnetically aligned single wall carbon nanotube films," Applied Physics Letters, vol. 77, No. 5, Jul. 31, 2000, pp. 666-668. cited by other .
Smith et al., "Structural anisotropy of magnetically aligned single wall carbon nanotube films," Applied Physics Letters, vol. 77, No. 5, Jul. 31, 2000, pp. 663-665. cited by other .
Andriotis et al., "Various bonding configurations of transition-metal atoms on carbon nanotubes: Their effect on contact resistance," Applied Physics Letters, vol. 76, No. 26, Jun. 26, 2000, pp. 3890-3892. cited by other .
Bezryadin et al., "Self-assembled chains of graphitized carbon nanoparticles," Applied Physics Letters, vol. 74, No. 18, May 3, 1999, pp. 2699-2701. cited by other .
Bezryadin et al., "Evolution of avalanche conducting states in electrorheological liquids," Physical Review E, vol. 50, No. 6, Jun. 1999, pp. 6896-6901. cited by other .
Lu et al., "Fullerene Pipes," Science, vol. 280, May 22, 1998, pp. 1253-1255. cited by other .
Yamamoto et al., "Orientation and purification of carbon nantubes using ac electrophoresis," J. Phys. D: Appl. Phys 31 (1998) L34-L36. cited by other .
Tohji et al., "Purifying single walled nanotubes," Nature, vol. 383, Oct. 24, 1996, p. 679. cited by other .
Dejan Rakovic, "Hierarchical Neural Networks and Brainwaves: Towards a Theory of Consciousness," Brain & Consciousness: Proc. ECPD Workshop (ECPD, Belgrade, 1997), pp. 189-204. cited by other .
Dave Anderson & George McNeill, "Artificial Neural Networks Technology," A DACS (Data & Analysis Center for Software) State-of-the-Art Report, Contract No. F30602-89-C-0082, ELIN: A011, Rome Laboratory RL/C3C, Griffiss Air Force Base, New York, Aug. 20, 1992. cited by other .
Greg Mitchell, "Sub-50 nm Device Fabrication Strategies," Project No. 890-00, Cornell Nanofabrication Facility, Electronics--p. 90-91, National Nanofabrication Users Network. cited by other .
John-William DeClaris, "An Introduction to Neural Networks," http://www.ee.umd.edu/medlab/neural/nn1.html. cited by other .
"Neural Networks," StatSoft, Inc., http://www.statsoftinc.com/textbook/stevnet.html. cited by other .
Stephen Jones, "Neural Networks and the Computation Brain or Maters relating to Artificial Intelligence," The Brain Project, http://www.culture.com.au/brain.sub.--proj/neur.sub.--net.htm. cited by other .
David W. Clark, "An Introduction to Neural Networks"; http://members.home.net/neuralnet/introtonn/index.htm. cited by other .
"A Basic introduction to Neural Networks"; http://blizzard.qis.uiuc.edu/htmldocs/Neural/neural.html. cited by other .
Meyer et al., "Computational neural networks: a general purpose tool for nanotechnology," Abstract, 5.sup.th Foresight Conference on Molecular Nanotechnology; http://www.foresight.org/Conferences/MNT05/Abstracts/Meyeabst.html. cited by other .
Saito et al., "A 1M Synapse Self-Learning Digital Neural Network Chip," ISSCC, pp. 6.5-1 to 6.5-10, IEEE 1998. cited by other .
Espejo, et al., "A 16 x 16 Cellular Neural Network Chip for Connected Component Detection," Jun. 30, 1999; http://www.imse.cnm.csic.es/Chipcat/espejo/chip-2.pdf. cited by other .
Pati et al., "Neural Networks for Tactile Perception," Systems Research Center and Dept. of Electrical Engineering, University of Maryland and U.S Naval Research Laboratory. 1987; http://www.isr.umd.edu/TechReports/ISR/1987/TR.sub.--87-123/TR.sub.--87-1- 23.phtml. cited by other .
Osamu Fujita, "Statistical estimation of the number of hidden units for feedforward neural networks," Neural Networks 11 (1998), pp. 851-859. cited by other .
Abraham Harte, "Liquid Crystals Allow Large-Scale Alignment of Carbon Nanotubes," CURJ (Caltech Undergraduate Research Journal), Nov. 2001, vol. 1, No. 2, pp. 44-49. cited by other .
"Quantum-Dot Arrays for Computation," ORNL Review vol. 34, No. 2, 2001, pp. 1-5. cited by other .
Jouri, M.A. et al., "Adaptive Analog VLSI Neural Systems," Chapman & Hall, London SE1 8HN, UK, 1996, pp. 92-95. cited by other .
Lipson et al., "Automatic Design and Manufacture of Robotic Lifeforms," NATURE, vol. 406, Aug. 31, 2000, pp. 974-978. cited by other .
E.S. Snow, et al., "Random networks of carbon nanotubes as electronic material", Applied Physics Letters, vol. 82, No. 12, Mar. 31, 2003, pp. 2145-2147. cited by other .
R. Martel, et al., "Ambipolar Electrical Transport in Semiconducting Single-Wall Carbon Nanotubes," Physical Review Letters, vol. 87, No. 25, Dec. 17, 2001, pp. 256805-1 to 256805-4. cited by other .
S. Heinze, et al., "Carbon Nanotubes as Schottky Barrier Transistors", vol. 89, No. 10, Sep. 2, 2002, pp. 106801-1 to 106801-4. cited by other .
M. Dubson, et al., "Measurement of the conductivity exponent in two-dimensional percolating networks: square lattice versus random-void continuum", Physical Review B, vol. 32, No. 11, Dec. 1, 1985, pp. 7621-7623. cited by other .
D.J. Frank, et al., "Highly efficient algorithm for percolative transport studies in two dimensions", Physical Review B, vol. 37, No. 1, Jan. 1, 1988, pp. 302-307. cited by other .
Uma R. Karmarkar, et al., "Mechanisms and significance of spike-timing dependent plasticity," Biol. Cybern. 87, 373-382 (2002), Jan. 28, 2002. cited by other .
Uma R. Karmarkar, et al., "A Model of Spike-Timing Dependent Plasticity: One or Two Coincidence Detectors?", J. Neurophysiol, vol. 88, pp. 507-513, Jul. 2002. cited by other .
M.C.W. van Rossum, et al., "Stable Hebbian Learning from Spkke-Timing-Dependent Plasticity", The Journal of Neuroscience, Dec. 1, 2003, 20(23), pp. 8812-8821. cited by other .
Xiaohui Xie, et al., "Spike-based learning rules and stabilization of persistent neural activity." cited by other .
Nace L. Golding, et al., "Dendritic spikes as a mechanism for cooperative long-term potentiation", NATURE, vol. 418, Jul. 18, 2002, pp. 326-330. cited by other .
Ozgur Turel, et al., "Possible nanoelectronic implementation of neuromorphic networks", Dept. o f Physics and Astronomy, Stony Brook University. cited by other .
V.C. Moore, et al., "Individually Suspended Single-Walled Carbon Nanotubes in Various Surfactants," Nano Letters, 2003, vol. 3; Sep. 9, 2003; American Chemical Society, pp. 1379-1382. cited by other .
J.M. Tour, et al., "NanoCell Electronic Memories," J.Am.Chem.Soc. 2003, 125, pp. 13279-13283. cited by other .
J. Zaumseil, et al., "Three-Dimensional and Multilayer Nanostructures Formed by Nanotransfer Printing," Nano Letters, 2003, vol. 3, No. 9, Jul. 31, 2003, American Chemical Society, pp. 1223-1227. cited by other .
Charles D. Schaper, "Patterned Transfer of Metallic Thin Film Nanostructures by Water-Soluble Polymer Templates," Nano Letters, 2003, vol. 3, No 9; Jul. 26, 2003, American Chemical Society, pp. 1305-1309. cited by other .
C.A. Dyke, et al., "Unbundled and Highly Functionalized Carbon Nanotubes from Aqueous Reactions," Nano Letters, 2003, vol. 3, No. 9; Aug. 19, 2003, American Chemical Society, pp. 1215-1218. cited by other .
J. Chung, et al., "Nanoscale Gap Fabrication by Carbon Nanotube-Extracted Lithography (CEL)", Nano Letters, 2003, vol. 3, No. 8; Jul. 9, 2003, American Chemical Society, pp. 1029-1031. cited by other .
O. Harnack, et al., "Rectifying Behavior of Electrically Aligned ZnO Nanorods," Nano Letters, 2003, vol. 3, No. 8; Jun. 24, 2003, American Chemical Society, pp. 1097-1101. cited by other .
M. S. Kumar, et al., "Influence of electric field type on the assembly of single walled carbon nanotubes," Chemical Physics Letters 383 (2004), Dec. 2, 2003; pp. 235-239. cited by other .
S.W. Lee, et al., "Dielectrophoresis and electrohydrodynamics-mediated fluidic assembly of silicon resistors," Applied Physics Letters, vol. 83, No. 18, Nov. 3, 2003, pp. 3833-3835. cited by other .
R. Krupke, et al., "Simultaneous Deposition of Metallic Bundles of Single-walled Carbon Nanotubes Using Ac-dielectrophoresis," Nano Letters, 2003, vol. 3, No. 8; Jul. 9, 2003; American Chemical Society, pp. 1019-1023. cited by other .
K. Bradley, et al., "Flexible Nanotube Electronics," Nano Letters, 2003, vol. 3, No. 10; Aug. 9, 2003, America Chemical Society, pp. 1353-1355. cited by other .
T.B. Jones, "Frequency-dependent orientation of isolated particle chains," Journal of Electrostatics, 25 (1990), Elsevier Science Publishers, pp. 231-244. cited by other .
L.A. Nagahara, "Directed placement of suspended carbon nanotubes for nanometer-scale assembly," Applied Physics Letters, vol. 80, No. 20; May 20, 2003; pp. 3826-3828. cited by other .
A. Bezryadin, et al., "Electrostatic trapping of single conducting nanoparticles between nanoelectrodes," Applied Physics Letters, 71 (9), Sep. 1, 1997, pp. 1273-1275. cited by other .
S. Suzuki, et al., "Quantitative Analysis of DNA Orientation in Stationary AC Electric Fields Using Fluorescence Anisotropy," IEEE Transactions of Industry Applications, vol. 34, No. 1; Jan./Feb. 1998, pp. 75-83. cited by other .
Phaedon Avouris, "Molecular Electronics with Carbon Nanotubes," Accounts of Chemical Research, vol. 35, No. 12; Jul. 31, 2002, pp. 1025-1034. cited by other .
X. Liu, et al., "Electric-Field Induced Accumulation and Alignment of Carbon Nanotubes," 2002 Annual Report Conference on Electrical Insulation and Dielectric Phenomena, pp. 31-34. cited by other .
R. Krupke, et al., "Contacting single bundles of carbon nanotubes with alternating electric fields," Appl. Phys. A. 76, Oct. 28, 2002, pp. 397-400. cited by other .
M. Law, et al., "Photochemical Sensing of NO.sub.2 with SnO.sup.2 Nanoribbon Nanosensors at Room Temperature," Angew. Chem. 2002, 114, Nr. 13, pp. 2511-2514. cited by other .
J. Tour, et al., "Nanocell Logic Gates for Molecular Computing," IEEE Transactions on Nanotechnology, vol 1, No. 2, Jun. 2002, pp. 100-109. cited by other .
A. Leonardi, et al., "Simulation methodology for dielectrophoresis in microelectronic Lab-on-a-chip," Modeling and Simulation of Microsystems 2002, pp. 96-99. cited by other .
J. Chung, et al., "Nanoscale Gap Fabrication and Integration of Carbon Nanotubes by Micromachining," Solid-State Sensor, Actuator and Microsystems Workshop, Jun. 2-6, 2003; Hilton Head Island, South Carolina, pp. 161-164. cited by other .
L. Zheng, et al., "Towards Single Molecule Manipulation with Dielectrophoresis Using Nanoelectrodes," IEEE-NANO 2003, Aug. 12-14, 2003, Moscone Convention Center, San Franscisco, CA; pp. 437-440. cited by other .
V.C. Moore, et al., "Individually Suspended Single-Walled Carbon Nanotubes in Various Surfactants," Nano Letters, 2003, vol. 3, No. 10; American Chemical Society; Sep. 8, 2003; pp. 1379-1382. cited by other .
R. Krupke, "Separation of Metallic from Semiconducting Single-Walled Carbon Nanotubes," Science, vol. 301; Jul. 18, 2003; pp. 344-347. cited by other .
Wolfgang Maass, "On the Relevance of Time in Neural Computation and Learning," In M. Li and A. Maruoka, editors, Proc. of the 8th International Conference on Algorithmic Learning Theory in Sendai (Japan), vol. 1316 of Lecture Notes in Computer Science, pp. 364-388. Springer (Berlin), 1997. cited by other .
Wolfgang Maass, "Noisy Spiking Neurons with Temporal Coding have more Computational Power than Sigmoidal Neurons," In M. Mozer, M. I. Jordan, and T. Petsche, editors, Advances in Neural Information Processing Systems, vol. 9, pp. 211-217. MIT Press (Cambridge), 1997. (pp. 1-13, including Appendix). cited by other .
L. Perrinet, et al., "Emergence of filters from natural scenes in a sparse spike coding scheme," Neurocomputing, 2003, pp. 1-14, http://www.laurent.perrinet.free.fr/publi/perrinet03neurocomputing.pdf. cited by other .
L. Perrinet, et al., "Coherence detection in a spiking neuron via Hebbian learning," Neurocomputing, 2002, vol. 44-46, No. C., pp. 817-822, http://www.laurent.perrinet.free.fr/publi/perrinet02.pdf. cited by other .
A. Jarosz, et al., "An Introductory Note on Gaussian Correlated Random Matrix," Feb. 21, 2003, pp. 1-20 http://www.if.uj.edu.pl/pl/koloSMP/prace/mdmatrix.pdf. cited by other .
K. Bradley, et al., "Influence of Mobile Ions on Nanotube Based FET Devices," Nano Letters, 2003, vol. 3, No. 5; American Chemical Society, Apr. 4, 2003; pp. 639-641. cited by other .
A. van Schaik, "Building blocks for electronic spiking neural networks," Neural Networks 14 (2001), pp. 617-628. cited by other .
A. Bezryadin, Trapping Single Particle with Nanoelectrodes, Physics News Graphics, Sep. 1997. cited by other .
Snow, et al., Nanofabrication with Proximal Probes, Proceedings of the IEEE, Apr. 1997. cited by other .
P. O'Connor, G. Gramegna, P. Rehak, F. Corsi, C. Marzocca, CMOS Preamplifier with High Linearity and Ultra Low Noise for X-Ray Spectroscopy, IEEE Transactions on Nuclear Science, vol. 44, No. 3, Jun. 1997, pp. 318-325. cited by other .
"Elements of Artificial Neural Networks" K. Mehrotra, C. K. Mohan, S. Ranka, 1997, MIT Press, pp. 116-135. cited by other .
"Self-assembled chains of graphitized carbon nanoparticles" A. Bezryadin, R.M. Westervelt, M. Tinkham, Dec. 21, 1998. cited by other .
"Collective Transport in Arrays of Small Metallic Dots" A. Alan Middleton, N.S. Wingreen, 1993, The American Physical Society, 0031-9007/93/71(19)/3198(4), pp. 3198 through 3201. cited by other .
Philip G. Collins & Phaedon Avouris, "Nanotubes for Electronics," Scientific American, Dec. 2000, pp. 62-69. cited by other .
Chen et al., "Aligning single-wall carbon nanotubes with an alternating-current electric field", Applied Physics Letters; Jun. 4, 2001--vol. 78, Issue 23, pp. 3714-3716. cited by other .
A. Bezryadin et al., "Evolution of Avalanche Conducting States in Electrorheological Liquids" Department of Physics and Division of Engineering and Applied Sciences, Harvard University, Cambridge, MA; Oct. 21, 1998, pp. 1-15. cited by other .
Yamamoto, et al. "Rapid Communication Orientation and Purification of Carbon Nanotubes Using AC Electrophoresis" J. Phys. D: Appl. Phys. 31 (1998) L34-L36. cited by other .
"Nanoparticles Get Wired" Dimes Institute, Delft University of Technology, Science Now, Jan. 28, 1997, p. 1. cited by other.

Primary Examiner: Vincent; David R.
Assistant Examiner: Tran; Mai T
Attorney, Agent or Firm: Lopez; Kermit D. Ortiz; Luis M. Ortiz & Lopez, PLLC

Parent Case Text

CROSS-REFERENCE TO RELATED APPLICATION

This patent application is a continuation-in-part of U.S. patent application Ser. No. 10/095,273 filed on Mar. 12, 2002 now U.S. Pat. No. 6,889,216, the disclosure of which is incorporated herein by reference. This patent application is also a continuation-in-part of U.S. patent application Ser. No. 10/162,524 filed on Jun. 5, 2002 now abandoned, the disclosure of which is incorporated herein by reference. This patent application is additionally a continuation-in-part of U.S. patent application Ser. No. 10/226,191 filed on Aug. 22, 2002 now abandoned, the disclosure of which is incorporated herein by reference.

Claims

The embodiments of an invention in which an exclusive property or right is claimed are defined as follows:

1. A system, comprising: a physical neural network configured utilizing nanotechnology and integrated with feedback circuitry, wherein said physical neural network comprises a plurality of nanoconductors comprising at least one of nanotubes, nanowires, or nanoparticles, suspended and free to move about in a dielectric medium and which form neural connections between pre-synaptic and post-synaptic components of said physical neural network; and a learning mechanism for applying Hebbian learning to said physical neural network.

2. The system of claim 1 wherein said learning mechanism utilizes a voltage gradient to implement Hebbian plasticity within said physical neural network.

3. The system of claim 1 wherein said learning mechanism utilizes voltage gradient dependencies associated with physical neural network to implement Hebbian learning within said physical neural network.

4. The system of claim 1 wherein said learning mechanism utilizes pre-synaptic and post-synaptic frequencies to provide Hebbian learning within said physical neural network.

5. The system of claim 1 wherein said learning mechanism utilizes a voltage gradient to implement anti-Hebbian plasticity within said physical neural network.

6. The system of claim 1 wherein said learning mechanism utilizes voltage gradient dependencies associated with physical neural network to implement anti-Hebbian learning within said physical neural network.

7. The system of claim 1 wherein said learning mechanism utilizes pre-synaptic and post-synaptic frequencies to provide anti-Hebbian learning within said physical neural network.

8. A system, comprising: a physical neural network configured utilizing nanotechnology and integrated with feedback circuitry, wherein said physical neural network comprises a plurality of nanoconductors comprising at least one of nanotubes, nanowires, or nanoparticles, suspended and free to move about in a dielectric medium and which form neural connections between pre-synaptic and post-synaptic components of said physical neural network; and a learning mechanism for applying Hebbian learning to said physical neural network wherein said learning mechanism utilizes a voltage gradient or pre-synaptic and post-synaptic frequencies thereof to implement Hebbian or anti-Hebbian plasticity within said physical neural network.

9. The system of claim 8 wherein said dielectric medium comprises a dielectric liquid.

10. The system of claim 9 wherein said plurality of nanoconductors form physical neural connections when said dielectric medium is exposed to an electric field, such that said physical neural connections can be strengthened or weakened depending upon a strengthening or weakening of said electric field or an alteration of a frequency thereof.

Description

TECHNICAL FIELD

The present invention generally relates to molecular technology, including nanotechnology. The present invention also relates to neural networks and neural computing systems and teaching methods thereof. The present invention also relates to Hebbian and anti-Hebbian learning techniques applicable to neural networks.

BACKGROUND OF THE INVENTION

Neural networks are computational systems that permit computers to essentially function in a manner analogous to that of the human brain. Neural networks do not utilize the traditional digital model of manipulating 0's and 1's. Instead, neural networks create connections between processing elements, which are equivalent to neurons of a human brain. Neural networks are thus based on various electronic circuits that are modeled on human nerve cells (i.e., neurons).

Generally, a neural network is an information-processing network, which is inspired by the manner in which a human brain performs a particular task or function of interest. Computational or artificial neural networks are thus inspired by biological neural systems. The elementary building blocks of biological neural systems are the neuron, the modifiable connections between the neurons, and the topology of the network.

Biologically inspired artificial neural networks have opened up new possibilities to apply computation to areas that were previously thought to be the exclusive domain of human intelligence. Neural networks learn and remember in ways that resemble human processes. Areas that show the greatest promise for neural networks, such as pattern classification tasks, speech and image recognition, are areas where conventional computers and data-processing systems have had the greatest difficulty.

In general, artificial neural networks are systems composed of many nonlinear computational elements operating in parallel and arranged in patterns reminiscent of biological neural nets. The computational elements, or nodes, are connected via variable weights that are typically adapted during use to improve performance. Thus, in solving a problem, neural net models can explore many competing hypothesis simultaneously using massively parallel nets composed of many computational elements connected by links with variable weights.

In contrast, with conventional von Neumann computers, an algorithm must first be developed manually, and a program of instructions written and executed sequentially. In some applications, this has proved extremely difficult. This makes conventional computers unsuitable for many real-time problems for which we have no efficient algorithm.

In a neural network, "neuron-like" nodes can output a signal based on the sum of their inputs, the output being the result of an activation function. In a neural network, there exists a plurality of connections, which are electrically coupled among a plurality of neurons. The connections serve as communication bridges among of a plurality of neurons coupled thereto. A network of such neuron-like nodes has the ability to process information in a variety of useful ways. By adjusting the connection values between neurons in a network, one can match certain inputs with desired outputs.

One does not program a neural network. Instead, one "teaches" a neural network by examples. Of course, there are many variations. For instance, some networks do not require examples and extract information directly from the input data. The two variations are thus called supervised and unsupervised learning. Neural networks are currently used in applications such as noise filtering, face and voice recognition and pattern recognition. Neural networks can thus be utilized as an advanced technique for processing information.

Neural networks that have been developed to date are largely software-based. A true neural network (e.g., the human brain) is massively parallel (and therefore very fast computationally) and very adaptable. For example, half of a human brain can suffer a lesion early in its development and not seriously affect its performance. Software simulations are slow because during the learning phase a standard computer must serially calculate connection strengths. When the networks get larger (and therefore more powerful and useful), the computational time becomes enormous.

For example, networks with 10,000 connections can easily overwhelm a computer. In comparison, the human brain has about 100 billion neurons, each of which can be connected to about 5,000 other neurons. On the other hand, if a network is trained to perform a specific task, perhaps taking many days or months to train, the final useful result can be built or "downloaded" onto a piece of hardware and also mass-produced. Because most problems requiring complex pattern recognition are highly specific, networks are task-specific. Thus, users usually provide their own, task-specific training data.

A number of software simulations of neural networks have been developed. Because software simulations are performed on conventional sequential computers, however, they do not take advantage of the inherent parallelism of neural network architectures. Consequently, they are relatively slow. One frequently used measurement of the speed of a neural network processor is the number of interconnections it can perform per second.

For example, the fastest software simulations available can perform up to approximately 18 million interconnects per second. Such speeds, however, currently require expensive super computers to achieve. Even so, approximately 18 million interconnects per second is still too slow to perform many classes of pattern classification tasks in real time. These include radar target classifications, sonar target classification, automatic speaker identification, automatic speech recognition, electrocardiogram analysis, etc.

The implementation of neural network systems has lagged somewhat behind their theoretical potential due to the difficulties in building neural network hardware. This is primarily because of the large numbers of neurons and weighted connections required. The emulation of even of the simplest biological nervous systems would require neurons and connections numbering in the millions and/or billions.

Due to the difficulties in constructing such highly interconnected processors, currently available neural network hardware systems have not approached this level of complexity. Another disadvantage of hardware systems is that they typically are often custom designed and configured to implement one particular neural network architecture and are not easily, if at all, reconfigurable in implementing different architectures. A true physical neural network chip, with the learning abilities and connectivity of a biological network, has not yet been designed and successfully implemented.

The problem with a pure hardware implementation of a neural network utilizing existing technology is the inability to physically form a great number of connections and neurons. On-chip learning can exist, but the size of the network is limited by digital processing methods and associated electronic circuitry. One of the difficulties in creating true physical neural networks lies in the highly complex manner in which a physical neural network must be designed and constructed. The present inventor believes that solutions to creating a true physical and artificial neural network lie in the use of nanotechnology and the implementation of a novel form of variable connections.

The term "Nanotechnology" generally refers to nanometer-scale manufacturing processes, materials and devices, as associated with, for example, nanometer-scale lithography and nanometer-scale information storage. Nanometer-scale components find utility in a wide variety of fields, particularly in the fabrication of microelectrical and microelectromechanical systems (commonly referred to as "MEMS"). Microelectrical nano-sized components include transistors, resistors, capacitors and other nano-integrated circuit components. MEMS devices include, for example, micro-sensors, micro-actuators, micro-instruments, micro-optics, and the like.

In general, nanotechnology presents a solution to the problems faced in the rapid pace of computer chip design in recent years. According to Moore's law, the number of switches that can be produced on a computer chip has doubled every 18 months. Chips now can hold millions of transistors. It is, becoming increasingly difficult, however, to increase the number of elements on a chip utilizing existing technologies. At the present rate, in the next few years the theoretical limit of silicon-based chips will likely have been attained. Because the number of elements and components that can be manufactured on a chip determines the data storage and processing capabilities of microchips, new technologies are required for the development of higher performance integrated circuit chips.

Present chip technology is also limited in cases where wires must be crossed on a chip. For the most part, the design of a computer chip is limited to two dimensions. Each time a circuit is forced to cross another circuit, another layer must be added to the chip. This increases the cost and decreases the speed of the resulting chip. A number of alternatives to standard silicon based complementary metal oxide semiconductor ("CMOS") devices have been proposed. The common goal is to produce logic devices on a nanometer scale. Such dimensions are more commonly associated with molecules than integrated circuits.

The issue of interconnects in neural network hardware poses a serious problem. Because of the massive interconnectivity, a neural network constructed with standard integrated electronic methods can never reach the desired neuron and synapse density, simply because the interconnections overwhelm the largely 2-diminsional chip. It can thus be appreciated that almost any sort of 3-diminsional connectivity, no matter how simple, could offer tremendous benefits.

Integrated circuits and electrical components thereof, which can be produced at a molecular and nanometer scale, include devices such as carbon nanotubes and nanowires, which essentially are nanoscale conductors ("nanoconductors"). Nanoconductors are tiny conductive tubes (i.e., hollow) or wires (i.e., solid) with a very small size scale (e.g., 0.7 to 300 nanometers in diameter and up to 1 mm in length). Their structure and fabrication have been widely reported and are well known in the art. Carbon nanotubes, for example, exhibit a unique atomic arrangement, and possess useful physical properties such as one-dimensional electrical behavior, quantum conductance, and ballistic electron transport.

Carbon nanotubes are among the smallest dimensioned nanotube materials with a generally high aspect ratio and small diameter. High-quality single-walled carbon nanotubes can be grown as randomly oriented, needle-like or spaghetti-like tangled tubules. They can be grown by a number of fabrication methods, including chemical vapor deposition (CVD), laser ablation or electric arc growth.

Carbon nanotubes can be grown on a substrate by catalytic decomposition of hydrocarbon containing precursors such as ethylene, methane, or benzene. Nucleation layers, such as thin coatings of Ni, Co, or Fe are often intentionally added onto the substrate surface in order to nucleate a multiplicity of isolated nanotubes. Carbon nanotubes can also be nucleated and grown on a substrate without a metal nucleating layer by using a precursor including one or more of these metal atoms. Semiconductor nanowires can be grown on substrates by similar processes.

Attempts have been made to construct electronic devices utilizing nano-sized electrical devices and components. For example, a molecular wire crossbar memory device can be constructed from crossbar arrays of nanowires sandwiching molecules that act as on/off switches. Such a device can be formed from a plurality of nanometer-scale devices, each device comprising a junction formed by a pair of crossed wires where a single wire crosses another and at least one connector species connects the pair of crossed wires in the junction. The connector species comprises a bi-stable molecular switch. The junction forms either a resistor or a diode or an asymmetric non-linear resistor. The junction has a state that is capable of being altered by application of a first voltage and sensed by the application of a second, non-destructive voltage. A series of related patents attempts to cover everything from molecular logic to how to chemically assemble these devices.

Such a molecular crossbar device has two general applications. The notion of transistors built from nanotubes and relying on nanotube properties is being pursued. Second, two wires can be selectively brought to a certain voltage and the resulting electrostatic force attracts them. When they touch, the Van der Walls force keeps them in contact with each other and a "bit" is stored. The connections in this apparatus can therefore be utilized for a standard (i.e., binary and serial) computer. The inventors of such a device thus desire to coax a nanoconductor into a binary storage media or a transistor. As it turns out, such a device is easier to utilize as a storage device.

A need exists for a physical neural network, which can be implemented in the context of a semiconductor integrated circuit (i.e., a computer chip). Such a device, which can be referred to as a "physical neural network chip" or a "synapse chip" is thus disclosed herein. Such a device, if successfully implemented would be well suited for use with neural networks.

Researchers in the neuro-biological fields have attempted to develop a computationally efficient algorithm that can emulate a biologically realistic neural network. Specifically, researchers have attempted to develop method and/or systems, which would allow the efficient calculation of Spike-Timing Dependent-Plasticity (STDP), while also permitting fully interconnected pulsed networks. In STDP, timing between pre- and post-synaptic events can cause a net potentiation (LTP) or a net depression (LTD) of synapses.

A background for STPD is explained in "Mechanisms and Significance of Spike-Timing Dependent Plasticity" by Uma R. Karmarkar, et al., Biol. Cybern. 87, 373-382 (2002). STPD is also explained generally in "A Model of Spike-Timing Dependent Plasticity: One or Two Coincidence Detectors?" by Uma R. Karmarkar, et al., J. Neurophysiol, 88: 507-513, 2002. An additional reference regarding STDP is "Spike-Based Learning Rules and Stabilization of Persistent Neural Activity," by Xiaohui Xie, et al., Dept. of Brain & Cog, Sci, MIT, Cambridge, Mass. A further reference regarding STDP and issues faced thereof is described in "Dendritic Spikes as a Mechanism for Cooperative Long-Term Potentiation," Nace L. Golding, et al., Nature, Vol. 418, 18 Jul. 2002, pp. 326-330. The aforementioned references are indicated herein for generally illustrative, background and edification purposes only and do not disclose any features which limit any embodiments disclosed herein.

In the field of neural networks, Hebbian and anti-Hebbian learning has been the subject of great interest in recent years. An example of Hebbian learning is disclosed in non-limiting and non-essential U.S. Pat. No. 6,643,627, "Dynamic Synapse for Signal Processing in Neural Networks," which issued to Liaw et al on Nov. 4, 2003, and which is incorporated herein by reference. Non-essential U.S. Pat. No. 6,643,627 is a non-essential reference with respect to embodiments of the present invention, and is referred to herein merely for general edification and background purposes only.

Another example of Hebbian learning is disclosed in non-limiting and non-essential U.S. Pat. No. 6,611,823, "Backlash Compensation Using Neural Network", which issued to Selmic et al on Aug. 26, 2003. U.S. Pat. No. 6,111,823 also is a non-essential reference with respect to embodiments of the present invention, and is referred to herein merely for general edification and background purposes only. A further example of Hebbian learning and rules thereof is disclosed in non-essential U.S. Pat. No. 6,601,049, "Self-adjusting multi-layer neural network architectures and methods therefore", which issued to David Cooper on Jul. 29, 2003, and is incorporated herein by reference. U.S. Pat. No. 6,601,049 is also a non-essential reference with respect to embodiments of the present invention, and is referred to herein merely for general edification and background purposes only.

To date, Hebbian and anti-Hebbian learning has been implemented utilizing software simulations and algorithms thereof. Some large scale physical neural network systems have also attempted to incorporate Hebbian and anti-Hebbian learning. Current Hebbian and anti-Hebbian learning systems have not been implemented via true physical neural networks, particularly networks developed at a molecular scale, such as physical neural networks implemented utilizing nanotechnology.

BRIEF SUMMARY OF THE INVENTION

The following summary of the invention is provided to facilitate an understanding of some of the innovative features unique to the present invention, and is not intended to be a full description. A full appreciation of the various aspects of the invention can be gained by taking the entire specification, claims, drawings, and abstract as a whole.

It is, therefore, one aspect of the present invention to provide for a physical neural network, including an adaptive neural network, which can be formed and implemented utilizing nanotechnology.

It is still another aspect of the present invention to provide a physical neural network, which can be formed from a plurality of interconnected molecular connections, such as, for example, molecules, nanoconnections, and/or nanoconnectors.

It is yet a further aspect of the present invention to provide a physical neural network, which can be formed from a plurality of molecules, including molecular conducting structures.

It a further aspect of the present invention to provide a physical neural network based on nanoconductors, such as, for example, nanowires and/or nanotubes.

It is also an aspect of the present invention to provide a nanotechnology-based physical neural network, which incorporates Hebbian and anti-Hebbian learning techniques and mechanisms thereof.

The above and other aspects can be achieved as will now be described. Methods and systems are disclosed herein in which a physical neural network can be configured utilizing nanotechnology. Such a physical neural network can compris