GUIDA TECNICA

Convoluzioni separabili in profondità

A depthwise separable convolution splits an ordinary convolution layer into two cheaper steps: a depthwise step that filters each channel on its own, and a 1 by 1 pointwise step that mixes channels.

Ascolta questa guidaNarrato dall'IA · ~8 min

In questa pagina9 minuti di lettura
  1. Panoramica
  2. Punti chiave
  3. Immersione profonda
  4. Counting the cost of one layer
  5. Implementazione nel mondo reale
  6. Il futuro delle convoluzioni separabili in profondità
  7. Metterlo in pratica
  8. Fonti e approfondimenti
  9. Continua a esplorare
  10. Domande frequenti

Panoramica

With 3 by 3 kernels it needs roughly 8 to 9 times less computation, which is why it underpins mobile vision networks such as MobileNet and Xception.

Punti chiave

  1. A depthwise separable convolution replaces one standard convolution with a per-channel depthwise filter followed by a 1 by 1 pointwise convolution.
  2. Its cost relative to a standard convolution is 1/N plus 1/k squared, about one ninth for 3 by 3 kernels.
  3. MobileNet's depthwise separable version lost about 1 point of ImageNet accuracy while using roughly one eighth of the multiply-adds of a full-convolution version.
  4. Laurent Sifre developed the operation at Google Brain in 2013, and Xception and MobileNets popularized it in 2016 and 2017.
  5. Depthwise layers have high memory traffic per operation, so real speedups depend on the hardware and runtime, not just operation counts.

Immersione profonda

A convolution layer in an image network slides small filters over a feature map, a grid of values with a height, a width and a number of channels. The input image has three channels, red, green and blue, and deeper layers commonly have hundreds of channels, each detecting some learned pattern. A standard convolution does two jobs at once. Each filter looks at a small spatial neighborhood, such as 3 by 3 pixels, and it looks at every input channel together, so a single filter both finds spatial patterns and combines channels. If there are M input channels and N output channels, the layer needs N separate filters, each of size 3 by 3 by M.

A depthwise separable convolution splits those two jobs. First, a depthwise convolution applies one small filter to each input channel separately, so channel 1 gets its own 3 by 3 filter, channel 2 gets its own, and so on, with no mixing between channels. Second, a pointwise convolution, which is a 1 by 1 convolution, combines the channels: at every pixel position it computes N weighted sums of the M depthwise outputs. The result has the same shape as a standard convolution's output, but the work is done in a cheaper, factorized way. The Xception paper notes that frameworks such as TensorFlow and Keras call this operation a separable convolution, which should not be confused with a spatially separable convolution that splits a 3 by 3 filter into 3 by 1 and 1 by 3 parts.

The savings follow from counting multiplications. For a feature map of size D_F by D_F, a standard convolution with a D_K by D_K kernel costs D_K x D_K x M x N x D_F x D_F multiply-adds. The depthwise step costs D_K x D_K x M x D_F x D_F, and the pointwise step costs M x N x D_F x D_F. Dividing the sum by the original gives 1/N + 1/D_K squared. With a 3 by 3 kernel and many output channels, that is just over one ninth, so the MobileNet paper describes the saving as 8 to 9 times less computation. Its worked example, an internal layer with 512 input and output channels on a 14 by 14 map, drops from 462 million multiply-adds and 2.36 million parameters to 52.3 million multiply-adds and 0.27 million parameters.

The idea has a short, well-documented history. According to the Xception paper, Laurent Sifre developed depthwise separable convolutions during a Google Brain internship in 2013, tested them in AlexNet, presented an overview at ICLR 2014 and reported detailed results in his thesis. Early Inception networks used one as their first layer. In October 2016 Francois Chollet posted Xception, which reinterpreted Inception modules as a step between ordinary and depthwise separable convolution and built an entire network from the latter. In April 2017 Andrew Howard and colleagues at Google posted MobileNets, which used depthwise separable layers everywhere except the first layer and added width and resolution multipliers to trade accuracy for speed. MobileNetV2, posted in January 2018, wrapped the depthwise step in an inverted residual block with a linear bottleneck. ConvNeXt, posted by Facebook AI Research in January 2022, showed that 7 by 7 depthwise convolutions help a modernized ResNet compete with vision transformers.

The accuracy cost is usually modest. MobileNet's own comparison found that a version built from full convolutions reached 71.7% ImageNet top-1 accuracy with 4,866 million multiply-adds and 29.3 million parameters, while the depthwise separable version reached 70.6% with 569 million multiply-adds and 4.2 million parameters: about 1 point of accuracy for roughly an eighth of the computation. Xception showed the reverse case: at a similar parameter count to Inception V3, building from depthwise separable layers improved ImageNet accuracy slightly and improved results more on Google's internal JFT dataset of 350 million images and 17,000 classes. Other alternatives to the standard convolution include grouped convolutions, which mix channels only within groups, and spatially separable kernels, but depthwise separable layers became the default building block of mobile vision models.

The main limitation is that fewer multiply-adds do not always mean faster code. The ShuffleNet V2 paper argued that speed also depends on memory access, and classified depthwise convolution with element-wise operations because it has a high ratio of memory access to computation. On hardware tuned for large matrix multiplications, such as data-center GPUs, a depthwise layer can therefore run well below the speed its operation count suggests. MobileNet's authors designed around this by putting 95% of computation time in 1 by 1 convolutions, which map directly to highly optimized matrix multiplication routines, with the depthwise layers holding about 3% of multiply-adds and 1% of parameters. A second limitation is capacity: depthwise filters have very few parameters, which is why MobileNet used little or no weight decay on them and MobileNetV2 expands channels before the depthwise step.

In practice, depthwise separable convolutions are free to use and built into every major framework. In PyTorch, a depthwise convolution is an ordinary Conv2d layer whose groups argument equals the number of input channels, followed by a Conv2d with a 1 by 1 kernel. TensorFlow and Keras offer it directly as a separable convolution layer. The main practical work is measuring real latency on the target device, because the theoretical saving only appears when the hardware and runtime handle depthwise layers efficiently.

04Esempio lavorato

Counting the cost of one layer

  1. Take a layer with a 14 by 14 feature map, 512 input channels, 512 output channels and a 3 by 3 kernel.

  2. Standard convolution: 3 x 3 x 512 x 512 x 14 x 14 is about 462 million multiply-adds, with 3 x 3 x 512 x 512, about 2.36 million weights.

  3. Depthwise step: 3 x 3 x 512 x 14 x 14 is about 0.9 million multiply-adds, with 4,608 weights.

  4. Pointwise step: 512 x 512 x 14 x 14 is about 51.4 million multiply-adds, with 262,144 weights.

  5. Add the two steps: about 52.3 million multiply-adds and about 0.27 million weights.

  6. Divide: 52.3 / 462 is about 0.113, matching 1/512 + 1/9.

Cosa mostra

The learner reproduces the MobileNet paper's Table 3 figures, about 8.8 times fewer multiply-adds and parameters, and sees that the pointwise step now holds almost all of the cost, which is why fast 1 by 1 convolution matters for these networks.

Implementazione nel mondo reale

On-phone image classification: MobileNetV2 reached 72.0% ImageNet top-1 accuracy with 3.4 million parameters and 300 million multiply-adds, running in about 75 milliseconds on one large core of a Google Pixel 1 phone with TensorFlow Lite.

Mobile object detection: the MobileNetV2 paper pairs the network with SSDLite, a detector that also uses depthwise separable layers, and reports it is 20 times more efficient and 10 times smaller than YOLOv2 while scoring higher on COCO.

Tiny models for constrained hardware: the original MobileNet paper's 0.25 width version has about 0.5 million parameters and 41 million multiply-adds, while still reaching 50.6% ImageNet accuracy.

Large-scale classification: Google's Xception replaced Inception modules with depthwise separable convolutions and, with slightly fewer parameters than Inception V3, scored 79.0% versus 78.2% top-1 on ImageNet.

Modern convolutional networks: ConvNeXt (2022), from Facebook AI Research, uses 7 by 7 depthwise convolutions to mix spatial information, separating spatial and channel mixing in a way its authors compare to vision transformers.

Il futuro delle convoluzioni separabili in profondità

Depthwise separable convolutions remain a standard building block, but the questions around them have shifted from operation counts to measured speed. The ShuffleNet V2 work in 2018 showed that memory access and fragmentation can dominate runtime in lightweight networks, and the practical answer has been to design models with latency measured on the real target device rather than estimated from multiply-adds. The operation has also found a role beyond mobile models. The 2022 ConvNeXt study reported that moving to 7 by 7 depthwise kernels raised its model's ImageNet accuracy from 79.9% to 80.6% at roughly constant computation, with no further gain beyond 7 by 7, and its authors compared depthwise convolution to the per-channel weighted sum in self-attention. That framing, separate operations for spatial mixing and channel mixing, now appears in both convolutional and transformer-style designs. Open problems include making depthwise layers efficient on accelerators built for dense matrix multiplication, choosing kernel sizes and expansion ratios for each layer automatically, and deciding when large-kernel depthwise convolutions or attention give the better accuracy for a given hardware budget. Each new generation of mobile and edge hardware changes those trade-offs, so published results apply to the devices and runtimes they were measured on.

Metterlo in pratica

Perché è importante

  • Costo e budget. Le decisioni relative all'architettura determinano prestazioni e costi operativi per anni.
  • Decisioni più chiare. La formazione tecnica aiuta i team a scegliere lo stack giusto, non solo quello più nuovo.
  • Controllo di qualità. Migliori scelte ingegneristiche riducono gli incidenti legati all’affidabilità nella produzione.

Attenzione

  • L'ottimizzazione di un benchmark può nascondere debolezze di sistema più ampie.
  • I costi delle infrastrutture e della manutenzione sono spesso sottostimati.
  • Le lacune in termini di sicurezza e osservabilità possono aumentare man mano che i sistemi diventano più complessi.

Prossimi passi

  1. Definire obiettivi di latenza, qualità e costi prima dell'implementazione.
  2. Benchmark in condizioni di carico e dati realistiche.
  3. Monitoraggio dello strumento per errori, deriva e impatto sull'utente.
  4. Preparare percorsi di rollback e risposta agli incidenti prima della scalabilità.

Fonti e approfondimenti

  1. arXivMobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
  2. arXivXception: Deep Learning with Depthwise Separable Convolutions
  3. arXivMobileNetV2: Inverted Residuals and Linear Bottlenecks
  4. arXivShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
  5. arXivA ConvNet for the 2020s
  6. PyTorch documentationtorch.nn.Conv2d

Continua a esplorare

Newsletter gratuita

Ricevi il briefing quotidiano sull'IA

Tre storie verificate sull'IA ogni mattina nei giorni feriali, scritte in un inglese semplice. Gratuito per sempre, senza pubblicità.

Un'email ogni giorno feriale. Disiscriviti con un clic. Non vendiamo né condividiamo mai il tuo indirizzo.

Mettiti alla prova

Take the Depthwise Separable Convolutions quiz

Feedback immediato su ogni risposta e un certificato condivisibile con un ID verificabile una volta superato un corso.

Inizia il quiz

Sostieni l'istruzione gratuita sull'IA. AI Understanding è un'organizzazione non profit 501(c)(3) — nessun annuncio, nessun paywall, mai. Fai una donazione

Domande frequenti

What is Depthwise Separable Convolutions?

A depthwise separable convolution splits an ordinary convolution layer into two cheaper steps: a depthwise step that filters each channel on its own, and a 1 by 1 pointwise step that mixes channels. With 3 by 3 kernels it needs roughly 8 to 9 times less computation, which is why it underpins mobile vision networks such as MobileNet and Xception.

Quali sono i due passaggi in cui una convoluzione separabile in profondità divide una convoluzione standard?

Il passaggio in profondità filtra ogni canale spazialmente in modo autonomo, mentre il passaggio puntuale 1x1 combina quindi le informazioni sui canali.

Nella fase approfondita, come vengono trattati i canali di ingresso?

La convoluzione in profondità applica un filtro spaziale separato a ciascun canale di ingresso senza mescolare i canali, il che la rende economica.

For many output channels, what operation-count reduction does the guide associate with a 3×3 depthwise separable convolution?

Per un kernel 3x3 il rapporto di risparmio si avvicina a 1/9 quando il numero di canali di uscita è elevato, consentendo circa da 8 a 9 volte meno operazioni.

Qual è il ruolo della convoluzione puntuale (1x1) in questo progetto?

La convoluzione puntuale 1x1 mescola i canali su ogni pixel, cosa che il passaggio in profondità ha deliberatamente evitato di fare.

Quale nota architettura ha reso popolari le convoluzioni separabili in profondità per i dispositivi mobili?

MobileNet è stato costruito attorno a convoluzioni separabili in profondità appositamente per consentire un'inferenza efficiente sui telefoni.