No thats fine, its what it goes like when its flatted, its this that'll provide the key for the next coat. Keep your wet'n'dry paper wet as well pal, to stop it getting clogged up and scoring the paint as your rubbing