Skip to main content
SpringerLink
Log in

CapsGAN: Using Dynamic Routing for Generative Adversarial Networks

  • Conference paper
  • First Online:
Advances in Computer Vision (CVC 2019)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 944))

Included in the following conference series:

Abstract

In this paper, we propose a novel technique for generating images in the 3D domain from images with high degree of geometrical transformations. By coalescing two popular concurrent methods that have seen rapid ascension to the machine learning zeitgeist in recent years: GANs (Goodfellow et al.) and Capsule networks (Sabour, Hinton et al.) - we present: CapsGAN. We show that CapsGAN performs better than or equal to traditional CNN based GANs in generating images with high geometric transformations using rotated MNIST. In the process, we also show the efficacy of using capsules architecture in the GANs domain. Furthermore, we tackle the Gordian Knot in training GANs - the performance control and training stability by experimenting with using Wasserstein distance (gradient clipping, penalty) and Spectral Normalization. The experimental findings of this paper should propel the application of capsules and GANs in the still exciting and nascent domain of 3D image generation, and plausibly video (frame) generation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/raeidsaqur/CapsGAN.

  2. 2.

    http://yann.lecun.com/exdb/mnist/.

References

  1. Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein GAN. arXiv preprint arXiv:1701.07875 (2017)

  2. Bergstra, J.S., Bardenet, R., Bengio, Y., Kégl, B.: Algorithms for hyper-parameter optimization. In: Advances in Neural Information Processing Systems, pp. 2546–2554 (2011)

    Google Scholar 

  3. Denton, E.L., Chintala, S., Fergus, R., et al.: Deep generative image models using a Laplacian pyramid of adversarial networks. In: Advances in Neural Information Processing Systems, pp. 1486–1494 (2015)

    Google Scholar 

  4. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)

    Google Scholar 

  5. Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., Courville, A.C.: Improved training of wasserstein GANs. In: Advances in Neural Information Processing Systems, pp. 5769–5779 (2017)

    Google Scholar 

  6. Hinton, G., Frosst, N., Sabour, S.: Matrix capsules with EM routing (2018)

    Google Scholar 

  7. Hinton, G.E., Krizhevsky, A., Wang, S.D.: Transforming auto-encoders. In: International Conference on Artificial Neural Networks, pp. 44–51. Springer (2011)

    Google Scholar 

  8. Im, D.J., Kim, C.D., Jiang, H., Memisevic, R.: Generating images with recurrent adversarial networks. arXiv preprint arXiv:1602.05110 (2016)

  9. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015)

  10. LeCun, Y., Huang, J.: The small NORB dataset, v1.0

    Google Scholar 

  11. Miyato, T., Kataoka, T., Koyama, M., Yoshida, Y.: Spectral normalization for generative adversarial networks. arXiv preprint arXiv:1802.05957 (2018)

  12. Odena, A., Olah, C., Shlens, J.: Conditional image synthesis with auxiliary classifier GANs. arXiv preprint arXiv:1610.09585 (2016)

  13. Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434 (2015)

  14. Sabour, S., Frosst, N., Hinton, G.E.: Dynamic routing between capsules. In: Advances in Neural Information Processing Systems, pp. 3859–3869 (2017)

    Google Scholar 

  15. Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X.: Improved techniques for training GANs. In: Advances in Neural Information Processing Systems, pp. 2234–2242 (2016)

    Google Scholar 

  16. Wu, J., Zhang, C., Xue, T., Freeman, B., Tenenbaum, J.: Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling. In: Advances in Neural Information Processing Systems, pp. 82–90 (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Toronto, Toronto, Canada

    Raeid Saqur & Sal Vivona

Authors
  1. Raeid Saqur
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sal Vivona
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Raeid Saqur or Sal Vivona .

Editor information

Editors and Affiliations

  1. Saga University, Saga, Saga, Japan

    Kohei Arai

  2. The Science and Information (SAI) Organization, Bradford, West Yorkshire, UK

    Supriya Kapoor

Appendices

Appendices

A Capsule Algorithms and Formulae

1.1 A.1 Squashing Function

$$\begin{aligned} v_j = \frac{||s_j||^2}{(1 + ||s_j||^2)} \frac{s_j}{||s_j||} \end{aligned}$$
(5)

Here, \(v_j\) is the vector output of capsule j and \(s_j\) is its total output.

1.2 A.2 Margin Loss

The following equation shows the margin-loss used for digit existence:

$$\begin{aligned} L_k = T_k\ max(0, m^+ - ||v_k||^2 + \lambda (1-T_k)\ max(0, ||v_k|| - m^{-})^2 \end{aligned}$$
(6)

\(\bullet \) Softmax Function

$$\begin{aligned} c_{ij} = \frac{exp(b_{ij})}{\sum _k exp(b_{ik})} \end{aligned}$$
(7)

1.3 A.3 Routing Algorithm

Fig. 6.
figure 6

Routing algorithm used in Capsnet as in [14].

Full size image

B Execution

1.1 B.1 Public Repository Details

All code pertaining to this research paper has been hosted on Github at author’s page: (https://raeidsaqur.github.io/CapsGAN/). All frameworks used and code execution information is available in README.md.

1.2 B.2 GPUs Used

For execution, we used one multi-GPU rig (with 2 NVIDIA Titan Xps) and Google Cloud Computing instance with P1000 GPU.

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Saqur, R., Vivona, S. (2020). CapsGAN: Using Dynamic Routing for Generative Adversarial Networks. In: Arai, K., Kapoor, S. (eds) Advances in Computer Vision. CVC 2019. Advances in Intelligent Systems and Computing, vol 944. Springer, Cham. https://doi.org/10.1007/978-3-030-17798-0_41

Download citation

Publish with us

Policies and ethics

Search

Navigation