If you haven't already, try adjusting it the opposite way; some red dots I have tried worked that way, ie. if you adjusted it down the red dot would move down, which would raise the point of impact.

The red dot acts like the front sight of iron sights.

So depending on what the "down" adjustment means: does it move the red dot down (poi up) or does it move the point of impact down (red dot up)?

The bottom line is to move the red dot to the point of impact, regardless of how the sight is marked.