My bearded dragon doesnt seem to like me.?

I handle my beardie on a regular basis and he just seems to not have much interest in me holding him...why is this? I just recently started only feeding him when i picking him up, will this work? He is about full grown is there sill time to tame him or am i pretty much stuck with a mean beardie?