How do I punish a boyfriend who is habitually late?

He loves to sleep, so whenever we have daytime plans, they end up getting scrapped because he sleeps in most of the time. He naps after work so he ends up calling me way too late (or not at all). Sometimes he even completely forgets we had plans. We've been together a year and a half. I have discussed this with him, how inconsiderate I think it is (within reason, of course I don't attack him every time), but he says to be patient and he'll improve. Of course that's a crock. What can I do to let him see how irritating and inconsiderate he is being?