Here are some thoughts and a roll up of what everyone has said:
1. Don't go out expecting to see or harvest anything. Kids, especially at that young age, can be fidgety and like to talk. If they don't realize what day it is (Saturday vs. Sunday) or if they don't know that you can't hunt on Sunday (if that is the law in your state) then take them hunting on a day, or in an area, that you don't normally hunt. That way you don't mess up your hunting area with noise and movement and you have a safe area to teach your young'un how to hunt. It is a pretend hunt to you but the real thing to them, and you will likely see small game anyway so they will have seen animals.
2. Take food and drink for them and for you. The idea of something special that they only get on hunts is a good one. Teach them how to eat quietly and to pack out their trash.
3. Keep them warm and dry. Pick the time of day that is best for that. Don't make them walk too far or stay out too long. Getting them their own set of kit (clothes, boots, little day pack with water bottle and snacks and flashlight) will make it more special for them, not to mention great photo opportunities.
4. You don't have to start with deer. Before deer hunting became the BIG thing in hunting most of us started out with small game (squirrels, rabbits, ground hogs, and so on). They will see more of them than deer due to the number of them in the woods and they can practice using binoculars, learn their habits, and so on faster than waiting for a deer to show itself. Also, taking a squirrel first will be a good way to gauge their feelings about taking an animals life, as in the mouse story above. Let them watch hunting videos with you and see how they react to a scene with an actual kill shot and maybe one where they animal does not die "immediately".
5. Definitely give them something quiet to do. I wouldn't do video games since they might turn the volume up (unless you can tape over the switch or disable the sound some way. A book or coloring book will be a good idea. Bring a blanket and pillow for when they get tired and want to take a nap.