Wedding Guests

Today Josh brought up the fact that he thinks we should invite a few "friends" that we don't regularly hang out with to our wedding. I said that we don't see them or hang out with them. He feels obligated/bad. Someone said that you should go all out and invite whoever you want for your wedding since it's going to be the only one you have. Makes sense, but.... we are poor?