Errichto's blog

By Errichto, 3 weeks ago, In English,

tl;dr — video tutorial https://www.youtube.com/watch?v=eMXNWcbw75E and codeforces GYM training https://codeforces.com/gym/102644 (register by finding this contest in GYM instead of using the link directly)
video editorial: part 1 (ABCDEF) and part 2 (GHI)
codes to all 9 problems: https://github.com/Errichto/youtube/tree/master/matrix-exponentiation

Prerequisites: binary exponentiation and iterative dp (you don't need to know matrices)

The youtube tutorial (link) focuses on intuition and graph-like visualization . Or, if you prefer, below is a shorter (less detailed) text tutorial instead. You can practice by solving a set of 9 educational problems in GYM https://codeforces.com/gym/102644. ABCD are easy, EF medium, GHI are hard. If you are stuck, see hints below or watch the full solution analysis — part 1 (ABCDEF) and part 2 (GHI).

Hints

C. Fibonacci
D. Count Paths
E. Knight Paths
F. Min Path
G. Recurrence With Square
H. String Mood Updates
I. Count Paths Qeuries

Matrix Exponentiation

Consider this problem:
String Mood — Limak can be either happy or sad. His mood changes (or stays same) when he reads an English uppercase letter. Letters S and D always make him sad, H makes him happy, and every vowel (AEIOU) flips his mood. Other letters do nothing.
Limak is happy right now. Among all $$$26^n$$$ possible strings of length $$$n$$$ ($$$n \leq 10^{18}$$$), count such strings that Limak will be happy after reading that string, modulo $$$10^9+7$$$.

If something can be solved with dp in $$$O(1)$$$ space, we can usually speed it up later with matrix exponentiation. This dp is easy — for length from $$$1$$$ to $$$n$$$ compute the number of strings making you happy and making you sad at the end.

dp in O(1) space

Let's visualize that by drawing vertices representing the two moods, and edges with the number of ways to move from one state to the other.

drawing 1

For example, if you are happy, there are 19 ways to make you happy and 7 ways to make you sad (SDAEIOU). Thin edges on the right represent the second letter of a string and the number there should be the same, because they are again 19 ways to make you happy if you were happy, etc. If we were asked about answer for $$$n = 2$$$, we would need to compute the number of ways to get from HAPPY state in first column to HAPPY state in third column, which is they yellow edge below.

drawing 2

That's $$$19 \cdot 19 + 7 \cdot 6 = 403$$$. In a similar way, we can compute all four counts for 2-letter strings: happy to happy (equal to $$$403$$$), happy to sad ($$$19\cdot7+7\cdot20=273$$$), sad to happy ($$$234$$$), sad to sad ($$$442$$$). We'll actually keep these four values $$$[[403,273],[234, 442]]$$$ in a 2-dimensional array, also called a matrix.

Starting from a matrix with four values describing 1-letter strings $$$[[19,7],[6,20]]$$$, in $$$O(1)$$$ we got a matrix describing 2-letter strings. We can do the same to get a matrix for 4-letter strings, then 8-letter strings, and so on. We can do binary exponentiation to find a matrix for any huge $$$n$$$. Formally, we compute $$$M^n$$$ where $$$M = [[19,7],[6,20]]$$$ and multiplying two matrices is exactly what we did above to compute a new matrix $$$[[403,273],[234, 442]]$$$, done with the following code:

for(int i = 0; i < s; i++) { // s is the number of states, s=2 in the String Mood problem
	for(int j = 0; j < s; j++) {
		for(int k = 0; k < s; k++) {
			new_matrix[i][k] += matrix1[i][j] * matrix2[j][k];
		}
	}
}

The total time complexity is $$$O(s^3 \cdot \log n)$$$ where $$$s$$$ is the matrix size (also called order), which is equal to the number of states (variables) you need in space-efficient dp. We had $$$s=2$$$ in String Mood so the complexity is just $$$O(\log n)$$$.

full C++ solution

Now, try to find the $$$n$$$-th Fibonacci number for $$$n \leq 10^{18}$$$ (problem link) by first implementing space-efficient dp with transitions of form new_variable += old_variable * x where $$$x$$$ is some number you will need to put in the initial matrix (like $$$19$$$ or $$$7$$$ in String Mood).

Footnote

Thanks to tnowak for suggesting and creating one problem, and to mnbvmar for a few problem ideas. For the future, I'm looking for somebody to help me with the preparation of such training contests. If you want to help and you are high-rated and perfectly with experience in using polygon, please write to me. I can pay you a little if you want.

I hope you learned something thanks to this tutorial. Feel free to discuss problems from the GYM contest or give links to more matrix exponentiation problems or useful resources. If anybody wants that, I can make tests public. If you are a teacher and want to use some problems in your classes, I will give you access in Polygon.

 
 
 
 
  • Vote: I like it
  • +584
  • Vote: I do not like it

»
2 weeks ago, # |
  Vote: I like it +4 Vote: I do not like it

I assume there is no way to register lately for such contest? Unfortunatly there seems to be no way to submit without registering.

Apart from that, thanks for the work, I can submit after contests ends, too.

  • »
    »
    2 weeks ago, # ^ |
      Vote: I like it +21 Vote: I do not like it

    Strange, it should be possible. Maybe go to GYM tab and find the contest there?

»
2 weeks ago, # |
  Vote: I like it 0 Vote: I do not like it

Great Initiative! Hoping for more such training contests. What topic are you planning next ?

»
2 weeks ago, # |
  Vote: I like it 0 Vote: I do not like it

I think you should make group or something if you are willing to organise more such contests :)

»
2 weeks ago, # |
  Vote: I like it +29 Vote: I do not like it

"I can pay you a little if you want". Respect for this helping attitude of yours.

  • »
    »
    7 days ago, # ^ |
      Vote: I like it +3 Vote: I do not like it

    It's good that you are appreciating his effort but please never say pay you, it kind of sounds disrespectful for the effort he has Put in. These are some of the situations where person did not care about money but rather just want to help others.

»
10 days ago, # |
  Vote: I like it 0 Vote: I do not like it

Kind of a stupid concern, but why does WA on pretest 1 give a -1? If someone has a WA on pretest 1 and then AC, he gets +1 in standings.

»
9 days ago, # |
  Vote: I like it +6 Vote: I do not like it

Waiting for the video :>

»
4 days ago, # |
Rev. 2   Vote: I like it 0 Vote: I do not like it

Can anyone suggest where I am wrong at I. Count Paths Qeuries it is still giving TLE


#include <bits/stdc++.h> using namespace std; #define ll long long #define rep(i,x,n,inc) for(i=x ; i<n ; i+=inc) #define hell 1000000007 const int sz = 32; void multiply(vector<vector<ll> > &a, vector<vector<ll> > &b, int s) { vector<vector<ll> >mul(s, vector<ll> (s, 0)); for (int i = 0; i < s; i++) { for (int j = 0; j < s; j++) { for (int k = 0; k < s; k++) { mul[i][j] += ((a[i][k] % hell) * (b[k][j] % hell)) % hell ; mul[i][j] %= hell; } } } for (int i = 0; i < s; i++) for (int j = 0; j < s; j++) a[i][j] = mul[i][j] % hell; } map < ll, vector<vector<ll> >>ma; int main() { ios::sync_with_stdio(0); cin.tie(0); cout.tie(0); ll t, z; int n, m, q, i, j, k, x, y; cin >> n >> m >> q; vector<vector<ll> > F(n, vector<ll> (n, 0LL)); rep(i, 0, m, 1) { cin >> x >> y; x--, y--; F[x][y] += 1; } ma[1LL] = F; rep(i, 1, sz, 1) { multiply(F, F, n); vector < vector<ll>> v; ma[1LL << i] = F; } while (q--) { int s, t, k; cin >> s >> t >> k; s--, t--; vector<vector<ll> > I(n, vector<ll> (n, 0LL)); rep(i, 0, n, 1) { I[i][i] = 1; // rep(j, 0, n, 1) cerr << M[i][j] << " "; cerr << "\n"; } rep(i, 0, sz, 1) { if ((1LL << i)&k) { multiply(I, ma[1LL << i], n); } } z = I[s][t] % 1000000007LL; cout << z << '\n'; } }
  • »
    »
    4 days ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    You are still doing matrix-matrix multiplications when answering queries. That will be too slow. You need to do vector-matrix multiplications. I.e. if you want to compute v*M*N*O*P, do (((v*M)*N)*O)*P instead of v*((((N*M)*O)*P).

    • »
      »
      »
      4 days ago, # ^ |
        Vote: I like it 0 Vote: I do not like it

      But first, you need to work out the vector v.

    • »
      »
      »
      4 days ago, # ^ |
      Rev. 2   Vote: I like it 0 Vote: I do not like it

      (((v*M)*N)*O)*P I am doing this only , what make you think I am doing v*((((N*M)*O)*P)

      By the way thank you very much I got AC by vector-matrix multiplication.

      • »
        »
        »
        »
        4 days ago, # ^ |
          Vote: I like it 0 Vote: I do not like it

        v (or your I) is a $$$n \times n$$$ matrix, and therefore each multiplication is $$$O(n^3)$$$. It needs to be a vector (so $$$1 \times n$$$). That way every multiplication is $$$O(n^2)$$$.

        • »
          »
          »
          »
          »
          4 days ago, # ^ |
            Vote: I like it 0 Vote: I do not like it

          Yes , I was able to score AC with your suggestions

      • »
        »
        »
        »
        4 days ago, # ^ |
          Vote: I like it 0 Vote: I do not like it

        The multiply() works in $$$O(n^3)$$$, right?

        You call multiply q*log(k) times, which makes it $$$O(n^3 * q * log(k))$$$

        Since q==n it is basically $$$O(n^4 * log(k))$$$

  • »
    »
    4 days ago, # ^ |
    Rev. 2   Vote: I like it +8 Vote: I do not like it

    Jakube is correct that you should do vector-matrix multiplication. And you should compute the time complexity next time before asking why your code is too slow.

    Also, your multiply function can be improved a lot:

    1. Don't use so many modulo operations, that's a waste of time and code. If a[i][k] is already stored modulo $$$p$$$, there's no need for (a[i][k]%p).
    2. To be more cache-friendly, you should change the order of for-loops so that the last for-loop wouldn't be the first dimension of cells you use. So, if you use a[i][k] and b[k][j], neither $$$i$$$ nor $$$k$$$ should be the third of three for-loops because you jump all over memory this way.

    Or you can just take implementation from my blog and compare the running time yourself.