So here is a typical section of the XML files:

    <DungeonRoomCountMin>
      <CurveOperation Type="Multiplication">
        <Curve Abscissa="Level">
          <CurvePoint X="1" Y="12" Link="true" />
          <CurvePoint X="8" Y="26" Link="true" />
          <CurvePoint X="11" Y="28" Factor="0" />
        </Curve>
        <Curve Abscissa="PlayerCount" BaseValue="1">
        </Curve>
      </CurveOperation>
    </DungeonRoomCountMin>

first, what is the "curve abscissa" section doing? Is it a linear output, that says, input one equals output 12, input eight equals output 26, so you have roughly 10+2X as the effective formula?

What does the "link" or "factor" parameters do?

What does it mean when I see a multiplication inside an addition inside a multiplication?

This math stuff is too deep for me. The devs never went into detail explaining this. What you are looking at in the stuff you copy pasted is the three points describing the curve that shows the minimum room count according to floor level. So if you were to draw the graph, the curve would start at Floor 1 having a minimum of 12 rooms and with that whatever abscissa math curve it would reach a minimum count of 26 at floor 8, Then the steepness of the curve changes so that from floor 8 to floor 11 the minium room count only changes to 28. Minimum nomber of rooms rise steep until floor 8, but by then have nearly reached max size and change only by a coupel of rooms until the last floor.

Don't mess with the Link stuff, I think it has to do with the Unity files. The XML files aren't all the game after all, just a few bits the devs left open for players to mess with. but you only want to change simple stuff so as not to break things. You could for example change the steepness of the curve by making that 8 mid point a 6 midpoint. Or you could increase or decrease the minimum room count (or max count). But I'd not touch the way the nature of the maths.

For example here is an old custom dungeon run I did back in 2014:

https://www.youtube.com/watch?v=WYjmukVfSTY&t=118s

I edited the max number of heroes in the party and the probability of events.

"That should be the line. Do note though that it will crash the game when you complete the floor. The elevator/end game screens have been hard-coded for a max of 4 heroes :("

Does that still apply?

Yeah. Since development on the game has stopped, nothing has changed.