I'm sure this has been asked and answered many times before, but the search function doesn't work. So...what does it mean for an improvement to have a cost of Infinite? How can it be built? Do I need to research a technology to make the cost less than infinite? If so, how do I know what tech to research for a given improvement?

I think it's probably either one of three reasons:

• It is a Unique Structure and/or Improvement and you have already built it elsewhere (you were building it simultaneously in two locations but one finished first).
• The thing you're building is non-terminating (it's supposed to go on forever unless cancelled). An example would be public 3D printing which would continually convert production to dust.
• You did something / lack something and you can no longer build / finish that thing.

:D