可微編程

出自維基百科,自由嘅百科全書

可微編程係一個編程範例,藉此數字計算機程序經由自動微分嘅方法微分得[1][2][3][4]通常藉由梯度下降,可以實現得基於梯度嘅程序參數優化。可微編程經已喺好多領域當中使用開,特別係喺科學計算(scientific computing)同埋人工智能當中。[4]

方法[編輯]

大多數可微編程框架係經由砌出一個圖表(graph)、之包含到程序中嘅控制流同埋數據結構嘅嚟工作。[5] 較早嘅嘗試通常分為兩類:

  • 基於靜態、編譯好嘅圖嘅方法,譬如TensorFlow[注 1] TheanoMXNet 。啲方法傾向於允許良好嘅編譯器優化同埋比較容易噉擴展去大型系統,但係佢哋嘅靜態性質限制唨交互能力同埋輕鬆創建得嘅程序類型(譬如啲涉及循環遞歸嘅),而且令到啲用戶好難去推敲(reason about)佢哋寫嘅程序。[5][6][7]
  • 基於運算符重載、動態圖嘅方法,如PyTorchAutoGrad 。佢哋嘅動態同埋交互性質令到大多數程序嘅編寫同埋推敲變得更加容易。但係,佢哋會提高解釋器開銷(尤其係在組成許多小操作時),可擴展性勩尐,而且編譯器優化冇益到佢哋乜嚌。 [6][7][4]

呢兩種早期方法都只能微分啲以適當方式為框架編寫嘅代碼,之限制唨佢哋戥其他程序嘅互相操作性。

最新軟件包 Zygote、之使Julia編程語言写嘅,同埋使用新編程語言嘅Myia ,佢哋透過捉語言嘅語法睇作圖而解決唨早時啲試驗當中撞到嘅問題。然之後任意代碼嘅中間表示部分都直接微分、優化同埋編譯得。 [5][8][6]

應用[編輯]

以下領域當中經已使用開可微編程,譬如將深度學習佮埋物理引擎用於機器人技術、可微分嘅光線追蹤圖像處理同埋概率編程[9][10][11][12][4]

睇埋[編輯]

[編輯]

  1. TensorFlow 1使用靜態圖方法,而TensorFlow 2默認使用動態圖方法。

[編輯]

  1. Baydin, Atilim Gunes; Pearlmutter, Barak; Radul, Alexey Andreyevich; Siskind, Jeffrey (2018). "Automatic differentiation in machine learning: a survey". Journal of Machine Learning Research. 18: 1–43.
  2. Wang, Fei; Decker, James; Wu, Xilun; Essertel, Gregory; Rompf, Tiark (2018), Bengio, S.; Wallach, H.; Larochelle, H.; Grauman, K. (編), "Backpropagation with Callbacks: Foundations for Efficient and Expressive Differentiable Programming" (PDF), Advances in Neural Information Processing Systems 31, Curran Associates, Inc.: 10201–10212, 喺2019-02-13搵到
  3. Innes, Mike (2018). "On Machine Learning and Programming Languages" (PDF). SysML Conference 2018. 原著 (PDF)喺2019年7月17號歸檔. 喺2021年2月14號搵到.
  4. 4.0 4.1 4.2 4.3 Innes, Mike; Edelman, Alan; Fischer, Keno; Rackauckas, Chris; Saba, Elliot; Viral B Shah; Tebbutt, Will (2019), ∂P: A Differentiable Programming System to Bridge Machine Learning and Scientific Computing, arXiv:1907.07587
  5. 5.0 5.1 5.2 Innes, Michael; Saba, Elliot; Fischer, Keno; Gandhi, Dhairya; Rudilosso, Marco Concetto; Joy, Neethu Mariya; Karmali, Tejan; Pal, Avik; Shah, Viral (2018-10-31). "Fashionable Modelling with Flux". arXiv:1811.01457 [cs.PL].
  6. 6.0 6.1 6.2 "Automatic Differentiation in Myia" (PDF). 原著 (PDF)喺2019-06-24歸檔. 喺2019-06-24搵到.
  7. 7.0 7.1 "TensorFlow: Static Graphs". 喺2019-03-04搵到.
  8. Innes, Michael (2018-10-18). "Don't Unroll Adjoint: Differentiating SSA-Form Programs". arXiv:1810.07951 [cs.PL].
  9. Degrave, Jonas; Hermans, Michiel; Dambre, Joni; wyffels, Francis (2016-11-05). "A Differentiable Physics Engine for Deep Learning in Robotics". arXiv:1611.01652 [cs.NE].
  10. "Differentiable Monte Carlo Ray Tracing through Edge Sampling". people.csail.mit.edu. 喺2019-02-13搵到.
  11. "SciML Scientific Machine Learning Open Source Software Organization Roadmap". sciml.ai. 喺2020-07-19搵到.
  12. "Differentiable Programming for Image Processing and Deep Learning in Halide". people.csail.mit.edu. 喺2019-02-13搵到.