It is possible to change just the lifters but as mentioned previously the cam and lifters wear together so should be replaced together.

Its really down to you, personally i would change the cam at the same time.