Simplified 32-bit is the older architecture of CPU's that also operating systems and programs was written for. For example 32-bit was/is limited to only be able to make use of 4GB RAM. theoretically the limit was 4,294,967,296 byte, but due to how RAM stick are constructed and Windows 32-bit utilized each byte, in reality it could only make use of 3.86 GB of RAM (if I remember correctly). Although I have many times seen people still install 16 GB or even 32 GB RAM on Windows 7, even when they had Windows 7 installed in 32-bit. More or less pointless and a waste of money (in more user scenarios). However if You had Windows 7 installed in 64-bit then it was working with that amount of RAM. 

Of course then there was a limitation on which Windows 7 You ran like, even if it was 64-bit. Home Basic only supported 8 GB RAM. Home Premium 16 GB RAM. Pro, Premium and Enterprise as much as 192 GB RAM. Still many Windows 7 Home users installed 32 GB RAM. Again more or less pointless. Unless there was some later Windows update that increased that limit in the lower versions ??? I can't remember all the details off the top of my head. It's been a while now since I toyed around with that.

Now we are on 64-bit for the average consumer. We have been for quite some time. Also meaning that programs and operating systems that are written for 64-bit can make use of the advantages that 64-bit brings to the table. As I said one such advantage (apart from being able to process more information at any given time) is being able to make use of more than 4 GB RAM (acctually more than 3.86 gb ram). 

As I said on a 32-bit operating system, or even if running a 32-bit application/program on a 64-bit system, it can't make use of more than 3.86GB RAM. So games and programs that hit that limit may crash to desktop. Game devs and engineers had to always take that limit into consideration when making a game and it could limit the creation of large living worlds for example, as if it needed more than 3.86GB at one time without emptying/releasing RAM inbetween, the game would crash. One thing that could cause such crashes was badly programmed/optimized games that just kept using and using RAM, not making it relase RAM before using more, until it hit the limit and crash. Also often called a memory leak. For example on an MMO RPG it could eman you ride around and lancspe loaded. In a well optimized game it would release RAM as You travelled and didnt need to have loaded all landscape, only the landsacpe You could see. In a bad optimized game it would load everything as You passed it and keep loading more and more even if You did not see it anymore on Your monitor, until it hit the RAM limit and crash to desktop. It's a very simplified explanation and I'm no engineer, but I think You get the picture.

Yes 64-bit Operating Systems often supports, by backwards compatibility/support, running 32-bit programs. 

On for example a newer Windows operating system (even if it's 64-bit), You can find for example a directory called "Program (x86)". That is because it still is made to be able to run older 32-bit apps/programs and the x86 architecture etc.

HOWEVER Apple decided (some time back) to drop 32-bit support in their newer version of their Operating Systems. That can cause and issue for Apple users if a program/app/client is written in 32-bit, if they have upgraded to the later version of the operating system.

I could go into more details as it's not really that simple and in fact 32-bit can be called x86 or x86-32. While 64-bit usually is called just x64, but can also be called x86-64, but hey that will only make it even more confusing to most, so let's skip that, he he he !!! :P

You've probably also heard 8-bit and 16-bit and that is even older and at this point almost ancient, so let's also leave that out in this post. :)

To me more technical, I quote: 

32-bit is a type of CPU architecture that is capable of transferring 32 bits of data per clock cycle. More plainly, it is the amount of information your CPU can process each time it performs an operation. You can think this architecture as a road that's 32 lanes wide; only 32 "vehicles" (bits of data) can go through an intersection at a time.

In more technical terms, this means processors can work with 32-bit binary numbers (decimal number up to 4,294,967,295). Anything larger and the computer would need to break the data into smaller pieces.

/End Quote 

Then You can probably figure out the advanatges of 64-bit as it's like a road with even more lanes for cars to drive in at the same time. Although it's not only doubled as far as RAM goes, but theoretically 64-bit has a limit of being able to use 18,446,744,073,709,551,616 (16 Exabytes) of RAM. 

Also worth noting that there is still a limit depending on which Windows 10 You use. 64-bit versions of Windows 10 Home is limited to "only" 128 GB. While 64-bit Windows 10 Pro, Enterprise, and Education supports up to 2 TB of RAM. However I would say 128 GB of RAM is more than sufficient for the average consumer. Even gamers. Even home content creators. It will be some time before we really need more. 8 GB used to be enough for gamers, but we are pushing that limit now and 16 GB is more and more becoming the standard. If do some content creation You should go for 32 GB or even 64 GB in extreme cases. If You need more and even 128 then You shouldn't really be on Windows 10 Home anyway, but use Pro.