Skip to content
Unverified Commit c8732dfa authored by Louis Del Valle's avatar Louis Del Valle Committed by GitHub
Browse files

Update sub_quadratic_attention.py

1. Determine the number of query chunks.
2. Calculate the final shape of the res tensor.
3. Initialize the tensor with the calculated shape and dtype, (same dtype as the input tensors, usually)

Can initialize the tensor as a zero-filled tensor with the correct shape and dtype, then compute the attention scores for each query chunk and fill the corresponding slice of tensor.
parent 8aa87c56
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment