Ifarabalẹ Laini ati Awọn adanwo Kernels Performer

Awọn idahun ati awọn alaye han bi o ṣe n mu ibeere naa.

1. Kí nìdí wo ni boṣewa softmax akiyesi asekale ibi pẹlu ọkọọkan ipari?
   • Akoko ati iranti rẹ dagba pẹlu onigun mẹrin ti ipari ọkọọkan
   • O nilo tito lẹsẹsẹ gbogbo awọn ami ami adibi
   • O le ṣe ilana aami kan nikan ni akoko kan
   • O nilo nẹtiwọọki nkankikan lọtọ fun ami-ami kan
2. Ohun-ini mathematiki wo ni o jẹ ki akiyesi laini yago fun matrix n-nipasẹ-n?
   • Commutativity ti afikun
   • Associativity ti matrix isodipupo
   • Ofin pinpin ti softmax
   • Eigenvalue ibajẹ
3. Kini ẹrọ FAVOR+ Performer's isunmọ?
   • Awọn imudojuiwọn isọdi Gradient
   • Awọn fokabulari tokenizer
   • Ekuro akiyesi softmax
   • Layer normalization statistiki
4. Kini idi ti Oluṣeṣe lo awọn ẹya aileto rere ju awọn trigonometric iṣaaju lọ?
   • Wọn yiyara lori awọn CPUs nikan
   • Wọn ṣe imukuro iwulo fun eyikeyi laileto
   • Wọn ṣe awoṣe ni kikun itumọ
   • Wọn ṣe iṣeduro awọn iwuwo akiyesi ti kii ṣe odi ati ilọsiwaju iduroṣinṣin nọmba
5. Kini idiju isunmọ ti akiyesi laini ara Oluṣe ni gigun ọkọọkan n?
   • O(n)
   • O(n^2)
   • O(n log n)
   • O(2^n)
6. Kini aropin ti o wọpọ ti akiyesi laini mimọ ni akawe si akiyesi softmax kikun?
   • Ko le ṣiṣẹ lori awọn GPU
   • Nigbagbogbo o ṣaṣeyọri didara awoṣe kekere diẹ
   • O ṣiṣẹ nikan fun awọn aworan
   • O nilo data ike fun gbogbo àmi
7. For attention written as phi(Q)(phi(K)^T V), which multiplication can be done first to avoid creating an n-by-n matrix?
   • Compute QK^T and store every pairwise score
   • Apply softmax to the full n-by-n matrix first
   • Compute phi(K)^T V before multiplying by phi(Q)
   • Multiply the final output by the sequence length
8. What benefit does using orthogonal random features provide in Performer’s FAVOR+ approximation?
   • It guarantees exact softmax for every sequence
   • It removes the need for query and key vectors
   • It makes attention quadratic in sequence length
   • It reduces variance in the random-feature estimate