Random sketch learning for deep neural networks in edge computing

Simple item page
dc.contributor.authorLi, Bin
dc.contributor.authorChen, Peijun
dc.contributor.authorLiu, Hongfu
dc.contributor.authorGuo, Weisi
dc.contributor.authorCao, Xianbin
dc.contributor.authorDu, Junzhao
dc.contributor.authorZhao, Chenglin
dc.contributor.authorZhang, Jun
dc.date.accessioned2021-03-26T10:27:47Z
dc.date.available2021-03-26T10:27:47Z
dc.date.issued2021-03-25
dc.description.abstractDespite the great potential of deep neural networks (DNNs), they require massive weights and huge computational resources, creating a vast gap when deploying artificial intelligence at low-cost edge devices. Current lightweight DNNs, achieved by high-dimensional space pre-training and post-compression, present challenges when covering the resources deficit, making tiny artificial intelligence hard to be implemented. Here we report an architecture named random sketch learning, or Rosler, for computationally efficient tiny artificial intelligence. We build a universal compressing-while-training framework that directly learns a compact model and, most importantly, enables computationally efficient on-device learning. As validated on different models and datasets, it attains substantial memory reduction of ~50–90× (16-bits quantization), compared with fully connected DNNs. We demonstrate it on low-cost hardware, whereby the computation is accelerated by >180× and the energy consumption is reduced by ~10×. Our method paves the way for deploying tiny artificial intelligence in many scientific and industrial applications.en_UK
dc.identifier.citationLi B, Chen P, Liu H, et al., (2021) Random sketch learning for deep neural networks in edge computing. Nature Computational Science, Volume 1, March 2021, pp. 221-228en_UK
dc.identifier.issn2662-8457
dc.identifier.urihttps://doi.org/10.1038/s43588-021-00039-6
dc.identifier.urihttps://dspace.lib.cranfield.ac.uk/handle/1826/16510
dc.language.isoenen_UK
dc.publisherSpringer Natureen_UK
dc.rightsAttribution-NonCommercial 4.0 International*
dc.rights.urihttp://creativecommons.org/licenses/by-nc/4.0/*
dc.titleRandom sketch learning for deep neural networks in edge computingen_UK
dc.typeArticleen_UK

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Random_sketch_learning_for_deep_neural_networks_in_edge-2021.pdf
Size:
1.09 MB
Format:
Adobe Portable Document Format
Description:
Download
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
1.63 KB
Format:
Item-specific license agreed upon to submission
Description:
Download

Collections

Staff publications (SATM)