1.3.3 bug fix

[n0b0dy007]

Continued a save game post update. For some reason (too many regions ... ~200 ..., I guess) it kept crashing in Tools.bsf/PickBag_Add() on an fixed-size array overrun check (line 921).
This was during end of turn Host.bsf/Host_Disperse_Slaves(), per the debugger stack trace.
Work-around was doubling the value of PICKBAG_ARRAY_LENGTH (Tools.bsf, line 881) to 16384.

--
"Premature optimization is the root of all evil."
D. Knuth and/or C. A. R. Hoare (a Heisenquote)

[n0b0dy007]

Looking at the code and context of Host.bsf/Host_Disperse_Slaves() some more, I decided to try to improve things.
1st was cleaning up the logic of Host_Disperse_Slaves_All(), which essentially decided on a threshold delta (per Faction) based on AI vs Human, and difficulty setting.
The revised version is just:

Code: Select all

FUNCTION Host_Disperse_Slaves_All()
{
	int count;
	int fac;
	int factionID;
	int mode;
	int delta;	// difference in # of slave POPs, to disperse

	mode = GetDiffMode();	// mode values SYS_ defined in MapGlobals.bsf

	count = GetNumFactions();
	for (fac = 0; fac < count; fac++)
	{
		factionID = GetFactionID(fac);
		if (Faction_IsWorldFactionOrInactive(factionID) == FALSE)
		{
			delta = mode + 1;	// at least one
			Host_Disperse_Slaves(factionID, delta);
		}
	}
}

[n0b0dy007]

2nd was looking closer at Host.bsf/Host_Disperse_Slaves(), which was doing some complicated figuring based on DS_SLAVE_COEFF_TOSTRUCT and such.
It finally dawned on me that the point of the exercise was to move Slave POPs from regions with too many, to regions with too few.
Not wanting to try solving the full k-partition problem (see: https://en.wikipedia.org/wiki/Partition_problem), I implemented a simple (weighted) load-balancing approach.
(Yes, it's O(3n) to the number of regions, but n is typically << REGION_MAX, which isn't all that large.)

First, the data structure:

Code: Select all

struct RegionCountsStruct
{
	int id;		// region id
	int nPOPs;	// total number of POPs
	int nSlaves;	// number of nPOPs that are slaves
	int nSlots;	// number of available structure slots, relative to then-current nPOPs
}
RegionCountsStruct localRegionCounts[REGION_MAX];	// worst-case: all regions owned by same faction

#define DS_RECEIVERS 0
#define DS_SENDERS 1

// treating parameter "delta" as the threshold between receiver # and sender # needed to effect transfer
FUNCTION Host_Disperse_Slaves(factionID, delta)
{
	int regCount;
	int rgn;
	int regionID;
	int nPOPs;	// total number of POPs for faction
	int nSlaves;	// total number of Slave POPs for faction
	int nSlots;	// total number of available structure slots for faction
	int avgSlaves;	// simple average of nSlaves / regCount
	int hwm;	// high water mark: largest # nSlaves seen in any region
	int iter;	// loop counter
	int diff;	// difference between region's nSlaves and faction's avgSlaves
	int destID;	// regionID of region receiving moved POP
	int popID;	// ID of POP selected for move
	int xfer;	// number of transfers to make

Then (after some pre-condition checks), the 1st pass:

Code: Select all

	// Pass 1: gather
	// also gather total number of POPs, slaves across all regions for this faction
	nPOPs = 0;
	nSlaves = 0;
	nSlots = 0;
	hwm = 0;
	for (rgn = 0; rgn < regCount; rgn++)
	{
		regionID = GetOwnedRegionID(factionID, rgn);
		localRegionCounts[rgn].id = regionID
		localRegionCounts[rgn].nPOPs = Region_Population_Count(regionID);
		localRegionCounts[rgn].nSlaves = Region_Population_CountEx(factionID, regionID, POP_SOCIAL_SLAVE);
		localRegionCounts[rgn].nSlots = Region_Structures_NumFreeSlot(regionID);
		// nSlots == 0 when # structures >= nPOPs
		// update running totals:
		nPOPs += localRegionCounts[rgn].nPOPs;
		nSlaves += localRegionCounts[rgn].nSlaves;
		nSlots += localRegionCounts[rgn].nSlots;
		hwm = Max(hwm,localRegionCounts[rgn].nSlaves);
	}

The 2nd pass decides sending and receiving regions, using simple load balancing, moving POPs from regions with "above average" (surplus) to regions "below average" (deficit).
Over successive turns, this has the effect of smoothing out the population to be ≈ the avg # in all regions.
Note that the nSlots field is currently unused, so the original "not disturb planification" logic is not present in this version.

Code: Select all

	avgSlaves = nSlaves / regCount; 	// n.b.: integer math
	avgSlaves = Max(avgSlaves, 1);		// make avg at least 1, so regions w/none are eligible as receivers
	TraceLogEx(gVerbosity.decisions, "", "decisions", -1, "faction %S: avg slaves# %d, max %d, total %d", gString, avgSlaves, hwm, nSlaves);

	// rely on bag's property to weight selections (see: Tools.bsf/PickBag_* and https://en.wikipedia.org/wiki/Set_(abstract_data_type)#Multiset)
	PickBag_ResetAndLock(DS_RECEIVERS);	// Bag 0 for receivers
	PickBag_ResetAndLock(DS_SENDERS);	// Bag 1 for senders

	// loop to classify regions as either receivers or senders, based on their # slaves relative to avgSlaves:
	for (rgn = 0; rgn < regCount; rgn++)
	{
		regionID = localRegionCounts[rgn].id;
		diff = localRegionCounts[rgn].nSlaves - avgSlaves;
		Region_Name2(regionID);	// see Region.bsf: sets gString2 (display via %S)
		TraceLogEx(gVerbosity.decisions, "", "decisions", -1, "region %S: slaves# %d (diff %d)", gString2, localRegionCounts[rgn].nSlaves, diff);

		if (diff < 0)		// Regions with deficit (i.e. # slaves below average) are candidates to receive<-
		{
			diff = -diff;	// make positive, for bag count
			PickBag_Add(DS_RECEIVERS, regionID, diff);		// larger deficit more likely to receive
			TraceLogEx(gVerbosity.decisions, "", "decisions", -1, "region %S: Adding as receiving candidate, slave deficit is %d", gString2, diff);
		}
		else if (diff > 0)	// Regions with excess (i.e. # slaves above average) can send->
		{
			PickBag_Add(DS_SENDERS, regionID, diff);	// larger surplus more likely to supply
			TraceLogEx(gVerbosity.decisions, "", "decisions", -1, "region %S: Adding as sending candidate, slave surplus is %d", gString2, diff);
		}
		// otherwise, already has avgSlaves - skip
	}

This design relies on the properties of the "bag"/multiset (https://en.wikipedia.org/wiki/Set_(abst ... )#Multiset) data structure to use the amount by which a region is over / under the average to make it more likely (than other regions) to send / receive transfers, respectively.

The final pass effects the actual transfer(s), if any:

Code: Select all

	// Relies on bag property that higher-count entries are more likely to be picked
	xfer = Max(delta, hwm - avgSlaves);	// excess to distribute, but at least the given delta
	xfer = Min(xfer, PickBag_Count(DS_SENDERS));	// but no more than available
	xfer = Min(xfer, PickBag_Count(DS_RECEIVERS));	// and no more than expected
	TraceLogEx(gVerbosity.decisions, "", "decisions", -1, "# of slaves to move: %d, %d sender(s), %d receiver(s)", xfer, PickBag_Count(DS_SENDERS), PickBag_Count(DS_RECEIVERS));
	// process while receiver(s)/sender(s) found (PickBag removes item, decreasing count)
	while (((PickBag_Count(DS_RECEIVERS) > 0) && (PickBag_Count(DS_SENDERS) > 0)) && (xfer > 0))
	{
		regionID = PickBag_PickEx(DS_SENDERS, TRUE, FALSE);	// pick a sender region, reducing its weight by one
		destID = PickBag_PickEx(DS_RECEIVERS, TRUE, FALSE);	// pick a receiver region, reducing its weight by one
		if ((regionID > 0) && (destID > 0) && (destID != regionID))
		{
			popID = Region_Population_PickOne(regionID, -1, POP_SOCIAL_SLAVE, FALSE);
			Region_Name2(regionID);	// see Region.bsf: sets gString2 (display via %S)
			Region_Name3(destID);	// see Region.bsf: sets gString3 (display via %S)
			TraceLogEx(gVerbosity.decisions, "", "decisions", -1, "%d: move slave ID %d from %S -> %S", xfer, popID, gString2, gString3);
			if (popID > 0)
			{
				xfer--;	// only count against total if the transfer happens
				Population_SetRegion(popID, destID);
				Message_Region_SlaveMove(factionID, regionID, destID)
			}
		}
		TraceLogEx(gVerbosity.decisions, "", "decisions", -1, "# of slaves to move: %d, %d sender(s), %d receiver(s)", xfer, PickBag_Count(DS_SENDERS), PickBag_Count(DS_RECEIVERS));
	}

The net result is as expected. Here's an excerpt from the decisions.log for one faction:

Code: Select all

Host_Disperse_Slaves(faction Bosporus, delta 2)
faction Bosporus: 13 region(s)
faction Bosporus: avg slaves# 2, max 3, total 29
region Phanagoria: slaves# 3 (diff 1)
region Phanagoria: Adding as sending candidate, slave surplus is 1
region Ampsalis: slaves# 0 (diff -2)
region Ampsalis: Adding as receiving candidate, slave deficit is 2
region Henochia: slaves# 3 (diff 1)
region Henochia: Adding as sending candidate, slave surplus is 1
region Azabitis: slaves# 3 (diff 1)
region Azabitis: Adding as sending candidate, slave surplus is 1
region Paniardis: slaves# 3 (diff 1)
region Paniardis: Adding as sending candidate, slave surplus is 1
region Corasus: slaves# 2 (diff 0)
region Tyrambae: slaves# 2 (diff 0)
region Vardanes: slaves# 2 (diff 0)
region Siracia: slaves# 2 (diff 0)
region Suruba: slaves# 0 (diff -2)
region Suruba: Adding as receiving candidate, slave deficit is 2
region Scymitae: slaves# 3 (diff 1)
region Scymitae: Adding as sending candidate, slave surplus is 1
region Abasgia: slaves# 3 (diff 1)
region Abasgia: Adding as sending candidate, slave surplus is 1
region Ebriapa: slaves# 3 (diff 1)
region Ebriapa: Adding as sending candidate, slave surplus is 1
# of slaves to move: 2, 7 sender(s), 4 receiver(s)
2: move slave ID 411304000 from Abasgia -> Ampsalis
# of slaves to move: 1, 6 sender(s), 3 receiver(s)
1: move slave ID 407634064 from Azabitis -> Suruba
# of slaves to move: 0, 5 sender(s), 2 receiver(s)

Checking the in-game output led to the discovery of another bug (see next post).

[n0b0dy007]

3rd, and final (for now) was fixing a minor bug in Messages.bsf/Message_Region_SlaveMove().
It adds variety to the messages displayed (at the "Average" setting, so you have to include that level when showing the message log), varying between IDS_DEC_SLAVE_MARKET_RESULT1, 2, and 3.
The actual text strings for these are defined in Data/Text/Text48.txt (and localized variants for French, German, and Spanish).
Sadly, the semantic sense of "Sender" and "Receiver" is reversed for IDS_DEC_SLAVE_MARKET_RESULT1, compared to the other two:

Code: Select all

IDS_DEC_SLAVE_MARKET_RESULT1, "%{0}u has sent slave workers to %{1}u.",
IDS_DEC_SLAVE_MARKET_RESULT2, "A wealthy merchant from %{0}u has bought slaves from %{1}u.",
IDS_DEC_SLAVE_MARKET_RESULT3, "A known noble from %{0}u has acquired slaves from %{1}u.",

This entailed special case logic to work around the bug:

Code: Select all

FUNCTION Message_Region_SlaveMove(factionID, srcID, destID)
{
	int choice;

	choice = Dice(3);
	Message_Reset(GUI_COLOR_DEFAULT);
	gMTP.factionID = factionID
	gMTP.regionID = destID
	gMTP.category = MSG_CAT_REGION
	gMTP.diffusion = MSG_DIF_EXPLICIT
	gMTP.importance = MSG_IMP_AVG
	gMTP.type = MSG_REGION_SLAVE_POSITIVE
	// @FIXME: text for 1 is semantically reversed from 2 and 3
	if (choice == 1)
	{
	gMTP.subInt[0] = srcID
	gMTP.subInt[1] = destID
	}
	else
	{
	gMTP.subInt[0] = destID
	gMTP.subInt[1] = srcID
	}
	gMTP.subTyp[0] = dataTypeRegion
	gMTP.subTyp[1] = dataTypeRegion
	sprintf(gMTP.resultStr, "IDS_DEC_SLAVE_MARKET_RESULT%d", choice);	
	Message_Store();
}

Changed files, and the decisions log file, enclosed in the attached SlaveDist.zip.

[n0b0dy007]

Another update on this. Upon further examination, it turns out the base game decision Events.bsf/Decision_Regional_Disperse_Slaves() (option 2) ends up calling this exact same Host.bsf/Host_Disperse_Slaves(), albeit with text promising to only consider regions with Slave Markets as eligible senders. (Also, the new DLC RGD Events.bsf/Decision_Regional_Disperse_Slaves() adds a traverse Region_PickNeighbour() to limit the distribution of "excess" slave POPs to just adjacent regions.)

So, with these in mind, I revised the logic of Host_Disperse_Slaves() further:
First, add a field "eligible" to RegionCountsStruct:

Code: Select all

struct RegionCountsStruct
{
	int id;		// region id
	int nPOPs;	// total number of POPs
	int nSlaves;	// number of nPOPs that are slaves
	int nSlots;	// number of available structure slots, relative to then-current nPOPs
	int eligible;	// whether region is eligible for slaves xfer
}

Second, add logic to set that value based on the criteria that the region contains a Slave Market, or is the Faction's capital, or is a Provincial capital:

Code: Select all

		localRegionCounts[rgn].nSlots = Region_Structures_NumFreeSlot(regionID);	// nSlots == 0 when # structures >= nPOPs
		// Flag if eligible for slaves xfer (prunes the search space):
		localRegionCounts[rgn].eligible = FALSE;
		if (Region_Structures_HasSlaveMarket(regionID) || (regionID == Faction_Politic_MainCapitalRgnID(factionID)) || Region_Province_IsLocalCapital(regionID))
		{
			localRegionCounts[rgn].eligible = TRUE;
		}

Then, use the value to prune the search space when considering eligible regions:

Code: Select all

	// loop to classify regions as either receivers or senders, based on their # slaves relative to avgSlaves:
	for (rgn = 0; rgn < regCount; rgn++)
	{
		if (localRegionCounts[rgn].eligible)
		{
			diff = localRegionCounts[rgn].nSlaves - avgSlaves

The net effect of this is to restore some player agency, where the building of Slave Markets influences the amount of slave POP movement.

Finally, I decided to make use of the region's number of available building slots. Regions with fewer POPs (incl. slaves) than structures (i.e. .nSlots < 0) create an additional demand:

Code: Select all

			// further weight by nSlots need, so those with deficit "catch up" by requesting slaves more often:
			if (localRegionCounts[rgn].nSlots < 0)
			{
				diff += localRegionCounts[rgn].nSlots;
			}

As a final tweak, regions that weren't otherwise selected (because they already have the average), but have no open build slots, are also added to the recipients pool for consideration:

Code: Select all

			else if (localRegionCounts[rgn].nSlots == 0)	// avg # slaves, but no open build slots either
			{
				PickBag_Add(DS_RECEIVERS, regionID, 1);		// getting a slave would open a build slot
				TraceLogEx(gVerbosity.decisions, "", "decisions", -1, "region %S: Adding as receiving candidate to open a build slot", gString2);
			}

Updated Map.bsf and decisions.log attached in SlaveDist2.zip.

[Pocus]

That's interesting, but aside from the message diversity, I think it goes to modding. Thanks in any case. I have changed for the new patch the faulty sentence to: IDS_DEC_SLAVE_MARKET_RESULT1, "%{0}u has received slave workers from %{1}u.",