People have to start from the ground, then first step, second, third and so on to reach their destination floor. Exactly the same way students have to start from Kindergarten, then grade one, grade two and three and so on to reach their math destination. Also, if some of the steps are broken in the staircase, it is still hard to reach the desired floor using those steps. Same way, if you are missing some of the basic concepts from elementary grades, math for you is still hard. Now, the kindergarten, first grade and second grade are like first couple of the steps of the stairs. You can learn this level of math easily, as you can jump enough to take yourself to second or third step of the stairs easily. As it is very hard to reach sixth or seventh step of a stairs by jumping from the ground, exactly the same way to learn grade five or higher grade math is very hard (or impossible) without having the good knowledge of the kindergarten to grade three or grade four math. Now, consider one person is jumping on the ground to reach the third floor of a building. Can this person make it? Never, if he is not Spiderman. For this person, to reach the third floor by jumping is impossible or very hard and finally he gave up saying that it was very hard to reach third floor.
Your aim should be to make math fun for your child and help them gain a solid understanding of the basics. Encourage your child to incorporate math into their daily activities, help them do mathematics worksheets, and then use the lessons therein to create fun activities for your child. The secret to learning math is to have an open mind about it. Just because you hated math doesn't mean your child should, and you will be doing them a disservice if you pass your negative feelings on to them. Make learning math fun, get a comprehensive set of mathematics worksheets to make it exciting and stimulating, and encourage your child's efforts all the way. Every math genius had to start with 1 + 1, just like your child.