People have to start from the ground, then first step, second, third and so on to reach their destination floor. Exactly the same way students have to start from Kindergarten, then grade one, grade two and three and so on to reach their math destination. Also, if some of the steps are broken in the staircase, it is still hard to reach the desired floor using those steps. Same way, if you are missing some of the basic concepts from elementary grades, math for you is still hard. Now, the kindergarten, first grade and second grade are like first couple of the steps of the stairs. You can learn this level of math easily, as you can jump enough to take yourself to second or third step of the stairs easily. As it is very hard to reach sixth or seventh step of a stairs by jumping from the ground, exactly the same way to learn grade five or higher grade math is very hard (or impossible) without having the good knowledge of the kindergarten to grade three or grade four math. Now, consider one person is jumping on the ground to reach the third floor of a building. Can this person make it? Never, if he is not Spiderman. For this person, to reach the third floor by jumping is impossible or very hard and finally he gave up saying that it was very hard to reach third floor.
But another person used the stairs to reach the same floor. This person found it very easy and reach there with little effort. Compare this person to a student who knows all the basic concepts learned in elementary grades. To learn grade six or grade seven math for this student will be easy. But there is another student in grade six and doesn't know the lower grade math concepts such as, times tables, factors or number system. This student is in the same situation as the person, who is jumping to reach third floor from the ground level. From the example it is very clear that mathematics in each grade have the same importance. So, you need to be focused on math in all the grades on all the basic concepts. Ask your teachers lots of questions. Keep asking until you are not clear about the concepts or topics you are working on.