You don't fit solid lifters on a hydraulic cam. That will munch the cam. You also shouldn't fit any other lifters once a cam and lifters have bedded in together. So swapping them about as you have will have only damaged the cam.
CVH's are never quiet by modern standards, always a tap here n there from the top end