Compare commits

...
Author SHA1 Message Date
Brian Neumann-Fopiano b0860f94fc Gap-free object rotation
- Rotations off the right angles fill every target cell from the nearest source block
- Source blocks that no target cell samples still place at their rotated position
- Right-angle rotations keep the direct transform
- Markers, canopy and tile data follow their resampled source blocks
- Datapack export bakes off-angle variants with the same resampler
- The unused rotateCopy paths are removed
- Placement goldens match the resampled output
2026-10-31 19:23:25 -04:00
Brian Neumann-Fopiano 2462330787 Transition water caps
- Saved and blended columns find solid layers that cap water at the water head
- Surface search passes over a water cap to the ground below it
- A capped column keeps its fluid height at the top of the cap
- A painted cap counts only where the blended column keeps the same blocks
- Saved terrain sources report their water surface
- Displacement fluid keys come from the first fluid at or below the fluid height
- Empty fluid keys never win the fluid blend
2026-10-31 19:23:25 -04:00
Brian Neumann-Fopiano c2eca3e6bd Hydrology climate zones
- Hydrology climate splits the land into wet and dry zones with a noise style
- Dry zones keep only the dryRainfall share of rainfall and pond density
- Zone edges ease across the transition band
- Policy climateZones exempts an area from the dry zones
- Policy fluid also fills streams and rivers that flow through the area
- Bank steepness reads the steepest natural ground between the bank and the water
- Bank edges roughen with smooth noise
- Natural ground heights reach the rasterizer at full precision
- Settings reject climate values outside 0..1
- Hydrology probes read the full precision natural heights
2026-10-31 19:23:25 -04:00
Brian Neumann-Fopiano 304e66b74b Version 4.3.7
- Iris version is 4.3.7 in gradle.properties
- Root build defaults to version 4.3.7
- Fabric, Forge and NeoForge builds default to version 4.3.7
2026-10-31 19:13:08 -04:00
Brian Neumann-Fopiano f1c82e8cec Saved world on create
- /iris create saves a new world before spawn preparation
- A crash before the first autosave keeps the world's seed
2026-10-31 13:22:37 -04:00
Brian Neumann-Fopiano 9339c0869e Saved world on create
- /iris create saves a new world before spawn preparation
- A crash before the first autosave keeps the world's seed
2026-10-31 12:39:29 -04:00
Brian Neumann-Fopiano bb32ea9d25 Datapack export
- /iris datapack export compiles a pack dimension into a 26.3 worldgen datapack
- Bukkit and mod loaders share the export arguments
- Pack compiler stages lower noise, terrain, caves, biomes, decoration, objects and structures to 26.3 resources
- Every approximation reports an export limit diagnostic
- Decorators and objects follow Iris placement rules for water, slope, height and support
- Procedural objects export their pre-built variants as templates
- Jigsaw exports share the template encoder and publisher with the worldgen export
- Both platforms expose vanilla data from the server classpath
- GoldenHashEngine and the export probes share one chunk spiral scan
- Headless 26.3 probes measure export cost and likeness against Iris
2026-10-31 03:41:40 -04:00
Brian Neumann-Fopiano 7ff4292013 Datapack export fidelity fixes
- Aquatic decorators place on the sea floor in water only
- Decorator waterlogging follows the placement site
- Templates never bake waterlogged blocks
- Object blocks outside the nominal box stay in the template
- Procedural objects export their pre-built variants as content-keyed templates
- Surface objects anchor on the ocean floor heightmap
- Surface objects reject leaf, log, or fluid ground under the origin
- Surface support checks test base corners with a slope-scaled drop
- Support ground search covers the full 16 block window
- MIN_HEIGHT, MAX_HEIGHT, and stilt modes sample terrain across every allowed rotation
- Procedural tree roots extend down to the ground
- Underwater objects need water and stay below the surface
- On-water objects anchor at the fluid surface
- Unsupported coral, sea pickles, and wall fans are left out of templates
- Object placements honor slopeCondition
- Two-block plants place both halves
- Shore decorators need solid ground below
- Decorator coverage matches the Iris style density
- Patchy decorator styles export as noise threshold patches
- Palette end weights round to the nearest count
- Ores in deepslate place their deepslate variants
- Pack description tells players to keep the Default pack selected
- Aquifer floods open sea cells near the fluid height
2026-10-31 03:36:32 -04:00
Brian Neumann-Fopiano 1ed1044474 Datapack export
- /iris datapack export compiles one pack dimension into a worldgen datapack for Minecraft 26.3 and later
- Bukkit and mod loaders take the same version, mode, namespace, output, format, objects and replace arguments
- The default overworld mode replaces the Overworld of a new world with the default world type
- Preset and dimension modes write a selectable world preset or a separate dimension
- The export runs in the background and publishes the zip or folder atomically
- Pack compiler stages turn noise, terrain, caves, subterrain, biomes, surface, decoration, objects and structures into a version-neutral IR
- Capability lowering fits every field to 26.3 density function ranges
- Disjoint min and max nodes fold to the input that always wins
- The 26.3 writer emits density functions, noise settings, biomes, features, structures, presets and dimensions
- Every export carries the Iris logo as its pack.png
- Unused objects, templates and features are pruned from the output
- The pruner keeps pack.png without reading references from its bytes
- Imported pack datapacks are bundled into the export
- Every approximation reports an export limit diagnostic
- Generation cost is counted against a node budget and reported as a warning
- Cave biomes over the wall palette limit use the closest kept palette
- Jigsaw exports share the template encoder, block states, destinations and publisher with the worldgen export
- DataVersion carries the pack format minor and world data version
- Both platforms expose vanilla data from the server classpath
- GoldenHashEngine and the export probes share one chunk spiral scan
- Headless 26.3 probes measure export cost and compare column and chunk likeness against Iris
- Real-codec tests decode the exported resources with the 26.3 registries
- Paper registry tests run in isolated registry scopes
- Command help and messages cover the export in every language file
2026-10-30 22:59:04 -04:00
Brian Neumann-Fopiano b00e28e42e Modded terrain write tests
- Tests cover fresh section writes in the modded chunk generator
2026-10-30 22:50:49 -04:00
Brian Neumann-Fopiano b61d2f305b Modded structure state key
- Modded biome policy supplies the structure state key from the generator
- History worlds key structure state by the active epoch
- Studio worlds get a fresh transient key
- Tests cover the key binding and the structure state codec
2026-10-30 22:50:49 -04:00
Brian Neumann-Fopiano 618bb28591 Fixed lattice eviction race
- Lattice eviction sorts a snapshot of tile access stamps
- Concurrent tile use during eviction no longer breaks the sort
- Test covers eviction under concurrent fills
2026-10-30 22:47:20 -04:00
Brian Neumann-Fopiano 6ceda7c53b Pregen optimization pass
- Hydrology height lattice stores samples in tiles with least-recently-used eviction
- Evicted lattice cells are no longer sampled twice
- Pond rim columns go through the lattice column cache
- Window planning reads its coarse nodes from the fine lattice
- Prefetch stays inside the pregeneration area while a pregeneration runs
- Sea inlet splat compares squared distances
- Bounds sampling answers a column from region constants when every reachable biome shares the bounds
- Base terrain heights are shared through a per-column memo
- Chunk water rasters use per-chunk palettes and a larger cache
- Raster channel lookups use a 4-block segment grid
- Chunks with no water in reach return the empty raster directly
- Riffle spreads stop at cells that cannot reach the chunk
- Cave pool terrain is built only for chunks with a pool site
- Concurrent raster builds for one chunk share a single build
- Section buffers resolve air once and support bulk column reads
- Cave decoration, perfection and transition capture read bounded column windows
- Generation history claims and saved biome writes elect one writer per batch
2026-10-30 22:34:31 -04:00
Brian Neumann-Fopiano b475d20aec Merge branch 'perf-gen' into perf-review 2026-10-30 21:42:42 -04:00
Brian Neumann-Fopiano e9c18cdc08 Merge branch 'perf-raster' into perf-review 2026-10-30 21:29:13 -04:00
Brian Neumann-Fopiano b10966af47 Merge branch 'perf-dem' into perf-review 2026-10-30 21:29:13 -04:00
Brian Neumann-Fopiano 8e625d4547 Merge branch 'perf-plan' into perf-review 2026-10-30 21:29:13 -04:00
Brian Neumann-Fopiano 0d148c69b5 Shared concurrent raster builds
- Made concurrent demands for one chunk wait for a single water raster build
- Added a test that a second demand reuses the build already running
2026-10-30 21:02:36 -04:00
Brian Neumann-Fopiano 806e847e90 Bounded column rereads
- Natural cave decoration reads columns in sixteen-row windows and drops the window after each decorated zone
- Perfection reads cells directly after a spike is normalized instead of rereading the column below it
2026-10-30 21:02:23 -04:00
Brian Neumann-Fopiano ed83c6fdeb Unqueued history claim waits
- Semantic claims and saved biome writes elect one writer per batch instead of queueing every caller on the monitor
- Waiting callers park until their batch settles and wake the next queued caller when the writer leaves
- Admission tests count parked followers as queued behind the history writer
2026-10-30 20:55:54 -04:00
Brian Neumann-Fopiano b2990104d2 Sped up hydrology sampling
- Bounds sampling answers a column from its region, land or sea type, or noise-picked biome when every biome it can still become shares the group bounds
- Base terrain heights are kept in a shared per-column memo for hydrology batches, terrain3D anchors and scalar sampling
- Hydrology height batches skip columns whose base height is already known
- The per-batch base height map and its thread local are removed
- The four inferred biome streams share one context region stream
- BakedStream exposes a baked-plane lookup without falling back to the source
- Tests cover region constants, the base height memo and bit-exact heights with and without region constants
2026-10-30 20:54:55 -04:00
Brian Neumann-Fopiano def73cff96 Clipped pregen prefetch
- Limited chunk prefetch rings to the windows a bounded pregeneration still needs
- Forwarded pregeneration region bounds from the async, medieval and modded methods
- Restored unclipped prefetch when the pregeneration closes
- Added a runtime test for clipped and restored prefetch
2026-10-30 20:54:55 -04:00
Brian Neumann-Fopiano ca33273d17 Sped up inlet splat
- Compared squared distances in SeaInletPlanner.splat and kept hypot for cells inside the rounding band
2026-10-30 20:54:54 -04:00
Brian Neumann-Fopiano 8773d4b008 Tiled hydrology lattice
- Stored lattice samples in compact 32x32 tiles with least-recently-used tile eviction
- Cached sampled pond rim columns in a block-resolution tile cache
- Read window coarse nodes from the fine lattice at stride 2
- Removed the window lattice and FineLatticeSource
- Sized the fine lattice for an eleven-window span and the region lattice for a seven-region span
- Updated lattice, planner and probe harness callers for the single fine lattice
2026-10-30 20:54:41 -04:00
Brian Neumann-Fopiano 1a914a404a Sped up column scans
- Section buffers resolve air once per buffer instead of on every empty read
- Section buffer writes remember the last palette entry they resolved
- Added bulk column reads to section buffers and the update-listening hunk
- Natural cave decoration, perfection and transition geometry capture read whole columns at once
- Iris execution contexts bind through a thread slot table instead of a ThreadLocal
- Semantic capture tracks resolved cave cells in a fixed bitmap instead of a BitSet
- Added tests for bulk column reads and for contexts when threads outnumber the slot table
2026-10-30 20:25:50 -04:00
Brian Neumann-Fopiano 1899de7f3d Deferred cave pool terrain
- Opened the cave pool terrain view only for chunks that draw a pool site
- Added a test that chunks without a pool site never open their caves
2026-10-30 20:09:02 -04:00
Brian Neumann-Fopiano 3c39092cc7 Pruned riffle distance spreads
- Stopped riffle distance spreads at cells that cannot reach the chunk
2026-10-30 20:09:02 -04:00
Brian Neumann-Fopiano 0c7adc5da4 Indexed raster channel lookups
- Indexed reach segments into a channel grid around each rasterized chunk
- Skipped sea inlet column checks outside each inlet's bounds
- Returned the empty raster for chunks with no water features in reach
- Added a test that covered channel lookups match the full segment scan
2026-10-30 20:09:02 -04:00
Brian Neumann-Fopiano a879985875 Cached more water rasters
- Stored raster tiers fluids and feature ids as per-chunk palettes
- Derived cave ceilings from terrain heights and water heads
- Raised the water raster cache from 1024 to 8192 chunks
2026-10-30 20:09:02 -04:00
498 changed files with 54571 additions and 1345 deletions

No files matched your search

@@ -0,0 +1,563 @@
package art.arcane.iris.probe;
import art.arcane.iris.pack.datapack.export.WorldgenDatapackExporter;
import art.arcane.iris.pack.datapack.export.WorldgenExportProgress;
import art.arcane.iris.pack.datapack.export.WorldgenExportRequest;
import art.arcane.iris.pack.datapack.export.WorldgenExportResult;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import net.minecraft.core.RegistryAccess;
import net.minecraft.core.registries.Registries;
import net.minecraft.resources.Identifier;
import net.minecraft.resources.ResourceKey;
import net.minecraft.world.level.dimension.LevelStem;
import net.minecraft.world.level.levelgen.Aquifer;
import net.minecraft.world.level.levelgen.Heightmap;
import net.minecraft.world.level.levelgen.NoiseGeneratorSettings;
import net.minecraft.world.level.levelgen.NoiseRouter;
import net.minecraft.world.level.levelgen.densityfunction.DensityFunction;
import java.io.File;
import java.io.IOException;
import java.io.PrintStream;
import java.lang.management.ManagementFactory;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.Enumeration;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.SortedMap;
import java.util.TreeMap;
import java.util.concurrent.Callable;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicReference;
import java.util.zip.ZipEntry;
import java.util.zip.ZipFile;
public final class DatapackCostProbe {
private static final String LOG_PREFIX = "[datapack-cost]";
static final String VANILLA_PRESET = "minecraft:normal";
private static final int WARMUP_ORIGIN = -4096;
private static final int MINIMUM_CALLS = 8;
private static final int TOP_OFFENDERS = 10;
private static final long ANALYSIS_STACK_BYTES = 1L << 30;
private static final double MEGABYTE = 1024.0 * 1024.0;
private static final double NANOS_PER_SECOND = 1_000_000_000.0;
private DatapackCostProbe() {
}
public static void main(String[] arguments) {
PrintStream diagnostics = System.err;
try {
execute(Configuration.parse(arguments));
} catch (Throwable failure) {
failure.printStackTrace(diagnostics);
diagnostics.flush();
System.exit(1);
}
System.out.flush();
System.exit(0);
}
private static void execute(Configuration configuration) throws Exception {
HeadlessNativeBootstrap.initialize();
Files.createDirectories(configuration.work());
Path work = Files.createTempDirectory(configuration.work(), "run-");
List<HeadlessNativeRegistries> loaded = new ArrayList<>(2);
try {
log("exporting " + configuration.dimension() + " from " + configuration.pack());
ExportRun export = export(configuration, work);
log(String.format(Locale.ROOT, "exported in %.1f s: %d files, %d density functions", export.stats().seconds(),
export.stats().files(), export.stats().densityFunctions()));
DensityJsonBreakdown.Report densityJson = onLargeStack("density-json",
() -> DensityJsonBreakdown.analyze(export.files(), configuration.dimension()).report());
Path datapack = work.resolve("datapack");
writeFiles(export.files(), datapack);
Path vanillaPack = Files.createDirectories(work.resolve("vanilla"));
ExportStats exportStats = export.stats();
export.files().clear();
long baseline = settledHeap();
long loadStarted = System.nanoTime();
HeadlessNativeRegistries vanilla = isolatedLoad(vanillaPack);
double vanillaLoad = (System.nanoTime() - loadStarted) / NANOS_PER_SECOND;
loaded.add(vanilla);
long vanillaHeap = settledHeap();
loadStarted = System.nanoTime();
HeadlessNativeRegistries exported = isolatedLoad(datapack);
double exportLoad = (System.nanoTime() - loadStarted) / NANOS_PER_SECOND;
loaded.add(exported);
long exportHeap = settledHeap();
log(String.format(Locale.ROOT, "registries loaded: vanilla %.2f s, export %.2f s", vanillaLoad, exportLoad));
WorldgenStageBench vanillaBench = new WorldgenStageBench(new WorldgenStageBench.Options("vanilla",
presetOverworld(vanilla.registries(), VANILLA_PRESET), vanilla.registries(), configuration.seed()));
warm(vanillaBench, configuration);
long vanillaSamplers = settledHeap();
WorldgenStageBench exportBench = new WorldgenStageBench(new WorldgenStageBench.Options("export",
exportedOverworld(exported.registries()), exported.registries(), configuration.seed()));
warm(exportBench, configuration);
long exportSamplers = settledHeap();
double loadStart = ManagementFactory.getOperatingSystemMXBean().getSystemLoadAverage();
Measurement vanillaMeasurement = new Measurement(vanillaBench);
Measurement exportMeasurement = new Measurement(exportBench);
measure(List.of(vanillaMeasurement, exportMeasurement), configuration);
double loadEnd = ManagementFactory.getOperatingSystemMXBean().getSystemLoadAverage();
Map<String, GeneratorStats> generators = new LinkedHashMap<>();
generators.put("vanilla", vanillaMeasurement.stats(vanillaLoad, vanillaHeap - baseline, vanillaSamplers - exportHeap));
generators.put("export", exportMeasurement.stats(exportLoad, exportHeap - vanillaHeap, exportSamplers - vanillaSamplers));
log("analyzing density function graphs");
Map<String, List<DensityCostAnalysis.RootCost>> roots = new LinkedHashMap<>();
roots.put("vanilla", onLargeStack("density-cost-vanilla", () -> analyzeRoots(vanillaBench.settings(), configuration.dimension())));
roots.put("export", onLargeStack("density-cost-export", () -> analyzeRoots(exportBench.settings(), configuration.dimension())));
Report report = new Report(configuration.pack().toString(), configuration.dimension(), configuration.seed(),
configuration.rounds() * configuration.side() * configuration.side(), configuration.warmup() * configuration.warmup(),
configuration.rounds(), new MachineLoad(Runtime.getRuntime().availableProcessors(), loadStart, loadEnd),
"single thread for chunk stages and calls, vanilla and export interleaved per round; registry load uses HeadlessNativeRegistries' 4-thread loader; export uses Iris's own pools",
"feature decoration not measured: Paper's applyBiomeDecoration needs a live ServerLevel and the headless feature world lives in test fixtures",
exportStats, generators, densityJson, roots, definitions());
writeReport(report, configuration.output());
System.out.println(table(report));
log("report written to " + configuration.output());
} finally {
for (HeadlessNativeRegistries registries : loaded) {
registries.close();
}
deleteTree(work);
}
}
private static ExportRun export(Configuration configuration, Path work) throws Exception {
Path exportRoot = Files.createDirectories(work.resolve("exports"));
StageClock clock = new StageClock();
WorldgenExportResult result;
long started;
long finished;
try (RealPackProbeSupport.Workspace workspace = RealPackProbeSupport.openWorkspace(new RealPackProbeSupport.WorkspaceOptions(
configuration.pack().toFile(), configuration.dimension(), LOG_PREFIX, new HeadlessNativeRuntime(), work))) {
WorldgenExportRequest request = WorldgenExportRequest.builder()
.pack(workspace.pack().toPath())
.dimension(configuration.dimension())
.exportRoot(exportRoot)
.objects(true)
.build();
started = System.nanoTime();
clock.start(started);
result = WorldgenDatapackExporter.create().export(request, clock);
finished = System.nanoTime();
}
if (!result.isSuccess()) {
throw new IllegalStateException("Export failed: " + result.message() + " " + result.diagnostics());
}
SortedMap<String, byte[]> files = zipEntries(result.output());
ExportStats stats = new ExportStats((finished - started) / NANOS_PER_SECOND, clock.seconds(finished), Files.size(result.output()),
files.size(), count(files, "/worldgen/density_function/"), bytes(files, "/worldgen/density_function/"),
count(files, "/worldgen/noise/"), count(files, "/worldgen/biome/"), result.diagnostics().size());
Files.delete(result.output());
return new ExportRun(stats, files);
}
private static void warm(WorldgenStageBench bench, Configuration configuration) {
long started = System.nanoTime();
bench.chunks(WARMUP_ORIGIN, WARMUP_ORIGIN, configuration.warmup(), new WorldgenStageBench.ChunkSamples());
WorldgenStageBench.CallBudget budget = new WorldgenStageBench.CallBudget(MINIMUM_CALLS * 4, MINIMUM_CALLS,
TimeUnit.SECONDS.toNanos(configuration.callSeconds()) / 4);
bench.uncachedBiome(budget, new WorldgenStageBench.CallSamples());
bench.baseHeight(budget, Heightmap.Types.WORLD_SURFACE_WG, new WorldgenStageBench.CallSamples());
bench.baseColumn(budget, new WorldgenStageBench.CallSamples());
log(String.format(Locale.ROOT, "%s warmed in %.1f s", bench.label(), (System.nanoTime() - started) / NANOS_PER_SECOND));
}
private static void measure(List<Measurement> measurements, Configuration configuration) {
int stride = configuration.side() + 4;
WorldgenStageBench.CallBudget budget = new WorldgenStageBench.CallBudget(
Math.max(1, configuration.calls() / configuration.rounds()), Math.max(1, MINIMUM_CALLS / configuration.rounds()),
TimeUnit.SECONDS.toNanos(configuration.callSeconds()) / configuration.rounds());
for (int round = 0; round < configuration.rounds(); round++) {
for (int index = 0; index < measurements.size(); index++) {
Measurement measurement = measurements.get((index + round) % measurements.size());
long started = System.nanoTime();
measurement.bench.chunks(round * stride, round * stride, configuration.side(), measurement.chunks);
measurement.bench.uncachedBiome(budget, measurement.uncachedBiome);
measurement.bench.baseHeight(budget, Heightmap.Types.WORLD_SURFACE_WG, measurement.surfaceHeight);
measurement.bench.baseHeight(budget, Heightmap.Types.OCEAN_FLOOR_WG, measurement.floorHeight);
measurement.bench.baseColumn(budget, measurement.column);
log(String.format(Locale.ROOT, "%s round %d measured in %.1f s", measurement.bench.label(), round,
(System.nanoTime() - started) / NANOS_PER_SECOND));
}
}
}
private static List<DensityCostAnalysis.RootCost> analyzeRoots(NoiseGeneratorSettings settings, String dimension) {
DensityCostAnalysis analysis = new DensityCostAnalysis(id -> DensityJsonBreakdown.category(dimension, id), TOP_OFFENDERS);
Map<String, DensityFunction> roots = new LinkedHashMap<>();
NoiseRouter router = settings.noiseRouter();
roots.put("temperature", router.temperature());
roots.put("vegetation", router.vegetation());
roots.put("continents", router.continents());
roots.put("erosion", router.erosion());
roots.put("depth", router.depth());
roots.put("ridges", router.ridges());
roots.put("chunk_surface_level", router.chunkSurfaceLevel());
roots.put("final_density", router.finalDensity());
if (settings.aquifers().isPresent()) {
Aquifer.Config aquifer = settings.aquifers().get();
roots.put("aquifer.barrier", aquifer.barrierNoise());
roots.put("aquifer.fluid_level_floodedness", aquifer.fluidLevelFloodednessNoise());
roots.put("aquifer.fluid_level_spread", aquifer.fluidLevelSpreadNoise());
roots.put("aquifer.lava", aquifer.lavaNoise());
roots.put("aquifer.exclusion", aquifer.exclusion());
roots.put("aquifer.surface_level", aquifer.surfaceLevel());
}
List<DensityCostAnalysis.RootCost> costs = new ArrayList<>(roots.size());
for (Map.Entry<String, DensityFunction> root : roots.entrySet()) {
costs.add(analysis.analyze(root.getKey(), root.getValue()));
}
return costs;
}
static LevelStem presetOverworld(RegistryAccess registries, String preset) {
return registries.lookupOrThrow(Registries.WORLD_PRESET)
.getOrThrow(ResourceKey.create(Registries.WORLD_PRESET, Identifier.parse(preset))).value().overworld()
.orElseThrow(() -> new IllegalStateException("World preset " + preset + " has no overworld"));
}
static LevelStem exportedOverworld(RegistryAccess registries) {
return registries.lookupOrThrow(Registries.LEVEL_STEM).getOrThrow(LevelStem.OVERWORLD).value();
}
private static HeadlessNativeRegistries isolatedLoad(Path directory) {
try (PaperRegistryScope scope = PaperRegistryScope.open()) {
return HeadlessNativeRegistries.load(directory);
}
}
private static long settledHeap() throws InterruptedException {
for (int pass = 0; pass < 4; pass++) {
System.gc();
Thread.sleep(200L);
}
return ManagementFactory.getMemoryMXBean().getHeapMemoryUsage().getUsed();
}
private static <T> T onLargeStack(String name, Callable<T> work) throws Exception {
AtomicReference<T> result = new AtomicReference<>();
AtomicReference<Throwable> failure = new AtomicReference<>();
Thread thread = new Thread(null, () -> {
try {
result.set(work.call());
} catch (Throwable problem) {
failure.set(problem);
}
}, name, ANALYSIS_STACK_BYTES);
thread.start();
thread.join();
if (failure.get() instanceof Exception exception) {
throw exception;
}
if (failure.get() instanceof Error error) {
throw error;
}
return result.get();
}
private static SortedMap<String, byte[]> zipEntries(Path zip) throws IOException {
SortedMap<String, byte[]> files = new TreeMap<>();
try (ZipFile archive = new ZipFile(zip.toFile())) {
Enumeration<? extends ZipEntry> entries = archive.entries();
while (entries.hasMoreElements()) {
ZipEntry entry = entries.nextElement();
if (!entry.isDirectory()) {
files.put(entry.getName(), archive.getInputStream(entry).readAllBytes());
}
}
}
return files;
}
private static void writeFiles(SortedMap<String, byte[]> files, Path directory) throws IOException {
Files.createDirectories(directory);
for (Map.Entry<String, byte[]> file : files.entrySet()) {
Path target = directory.resolve(file.getKey()).normalize();
if (!target.startsWith(directory)) {
throw new IllegalArgumentException("Export file escapes the datapack root: " + file.getKey());
}
Files.createDirectories(target.getParent());
Files.write(target, file.getValue());
}
}
private static int count(SortedMap<String, byte[]> files, String folder) {
int count = 0;
for (String path : files.keySet()) {
if (path.contains(folder) && path.endsWith(".json")) {
count++;
}
}
return count;
}
private static long bytes(SortedMap<String, byte[]> files, String folder) {
long bytes = 0;
for (Map.Entry<String, byte[]> file : files.entrySet()) {
if (file.getKey().contains(folder) && file.getKey().endsWith(".json")) {
bytes += file.getValue().length;
}
}
return bytes;
}
private static void writeReport(Report report, Path output) throws IOException {
Files.createDirectories(output.getParent());
Gson gson = new GsonBuilder().setPrettyPrinting().serializeSpecialFloatingPointValues().create();
Files.writeString(output, gson.toJson(report), StandardCharsets.UTF_8);
}
private static Map<String, String> definitions() {
Map<String, String> definitions = new LinkedHashMap<>();
definitions.put("chunks", "per stage: wall ms and current-thread CPU ms per chunk; rounds of side x side chunks alternate vanilla and export");
definitions.put("chunks.biomes", "ChunkGenerator.createBiomes on a fresh ProtoChunk, ms per chunk over the grid plus a one-chunk ring");
definitions.put("chunks.noise", "NoiseChunk construction (aquifer init) plus NoiseBasedChunkGenerator.doFill, ms per chunk");
definitions.put("chunks.surface", "MaterialSystem.buildSurface with the fill's NoiseChunk and a region biome manager, ms per chunk");
definitions.put("chunks.carverBiome", "one uncached BiomeSource lookup at the chunk origin, as ChunkAccess.carverBiome memoizes per chunk, ms per chunk");
definitions.put("uncachedBiome", "BiomeSource.createUncachedResolver(...).getNoiseBiome at random quart positions within 32k blocks, wall us per call and mean thread CPU us per call");
definitions.put("baseHeight", "NoiseBasedChunkGenerator.getBaseHeight as structure placement calls it, us per call");
definitions.put("baseColumn", "NoiseBasedChunkGenerator.getBaseColumn, us per call");
definitions.put("retainedHeapBytes", "used heap after four System.gc passes, delta across the registry load");
definitions.put("samplerHeapBytes", "used heap delta across RandomState creation and warmup (compiled samplers and buffer pools)");
definitions.put("roots.dagNodes", "distinct non-constant density function and spline nodes reachable, references shared");
definitions.put("roots.cachedNodes", "non-constant evaluations per point with caches enabled: every distinct cache body once, all branches");
definitions.put("roots.expandedNodes", "non-constant evaluations per point with every reference and cache re-expanded, all branches");
definitions.put("roots.uncachedPointWorst", "worst single uncached point evaluation with lazy splines, interval_select and range_choice; interpolated off-lattice costs 8 corners at expanded cost; find_top_surface counts its probe steps");
definitions.put("roots.dagShare", "share of the distinct nodes by the category of the named file that holds them; offset copies are Grad2 and Taps duplicates");
definitions.put("roots.exclusive", "named density functions ranked by multiplicity x own nodes in the expanded tree");
definitions.put("roots.inclusive", "named density functions ranked by multiplicity x subtree nodes in the expanded tree");
definitions.put("densityJson.structurallyDuplicateBytes", "bytes beyond one file per JSON shape, with numbers, density function references and noise ids abstracted");
return definitions;
}
private static String table(Report report) {
StringBuilder out = new StringBuilder(4096);
GeneratorStats vanilla = report.generators().get("vanilla");
GeneratorStats export = report.generators().get("export");
out.append(String.format(Locale.ROOT, "%s %s seed %d: %d chunks in %d interleaved rounds after %d warmup chunks, one thread, load %.0f-%.0f on %d cores%n",
LOG_PREFIX, report.dimension(), report.seed(), report.chunks(), report.rounds(), report.warmupChunks(),
report.load().loadAverageStart(), report.load().loadAverageEnd(), report.load().processors()));
out.append(String.format(Locale.ROOT, "%-34s %10s %10s %8s %10s %10s %8s%n", "stage (mean)", "vanilla", "export", "ratio",
"cpu van", "cpu exp", "ratio"));
out.append(String.format(Locale.ROOT, "%-34s %10s %10.1f%n", "export (s)", "-", report.export().seconds()));
row(out, "registry load (s)", vanilla.registryLoadSeconds(), export.registryLoadSeconds());
row(out, "retained heap after load (MB)", vanilla.retainedHeapBytes() / MEGABYTE, export.retainedHeapBytes() / MEGABYTE);
row(out, "sampler heap after warmup (MB)", vanilla.samplerHeapBytes() / MEGABYTE, export.samplerHeapBytes() / MEGABYTE);
stage(out, "biome fill (ms/chunk)", vanilla.chunks().biomes(), export.chunks().biomes());
stage(out, "noise fill (ms/chunk)", vanilla.chunks().noise(), export.chunks().noise());
stage(out, "surface (ms/chunk)", vanilla.chunks().surface(), export.chunks().surface());
stage(out, "carver biome lookup (ms/chunk)", vanilla.chunks().carverBiome(), export.chunks().carverBiome());
row(out, "chunk total (ms/chunk)", vanilla.chunks().totalMean(), export.chunks().totalMean(),
vanilla.chunks().totalCpuMean(), export.chunks().totalCpuMean());
call(out, "uncached getNoiseBiome (us/call)", vanilla.uncachedBiome(), export.uncachedBiome());
call(out, "getBaseHeight WORLD_SURFACE (us)", vanilla.baseHeightSurface(), export.baseHeightSurface());
call(out, "getBaseHeight OCEAN_FLOOR (us)", vanilla.baseHeightFloor(), export.baseHeightFloor());
call(out, "getBaseColumn (us/call)", vanilla.baseColumn(), export.baseColumn());
out.append(String.format(Locale.ROOT, "%-24s %9s %9s %9s %10s %10s %10s %10s%n", "router root", "dag(v)", "dag",
"cached", "expand(v)", "expanded", "uncPt(v)", "uncachedPt"));
List<DensityCostAnalysis.RootCost> vanillaRoots = report.roots().get("vanilla");
List<DensityCostAnalysis.RootCost> exportRoots = report.roots().get("export");
for (DensityCostAnalysis.RootCost root : exportRoots) {
DensityCostAnalysis.RootCost reference = find(vanillaRoots, root.root());
out.append(String.format(Locale.ROOT, "%-24s %9s %9s %9s %10s %10s %10s %10s%n", root.root(),
reference == null ? "-" : compact(reference.dagNodes()), compact(root.dagNodes()), compact(root.cachedNodes()),
reference == null ? "-" : compact(reference.expandedNodes()), compact(root.expandedNodes()),
reference == null ? "-" : compact(reference.uncachedPointWorst()), compact(root.uncachedPointWorst())));
}
out.append("top expanded offenders (exclusive share of root, multiplicity)\n");
for (DensityCostAnalysis.RootCost root : exportRoots) {
if (root.exclusive().isEmpty() || root.expandedNodes() < 1.0e4) {
continue;
}
StringBuilder line = new StringBuilder(String.format(Locale.ROOT, " %-22s", root.root()));
for (int index = 0; index < Math.min(2, root.exclusive().size()); index++) {
DensityCostAnalysis.Offender offender = root.exclusive().get(index);
line.append(String.format(Locale.ROOT, " %4.1f%% %s x%s;", 100.0 * offender.contribution() / root.expandedNodes(),
shortName(offender.name()), compact(offender.multiplicity())));
}
out.append(line).append('\n');
}
for (DensityCostAnalysis.RootCost root : exportRoots) {
if (root.dagNodes() < 1.0e4) {
continue;
}
List<Map.Entry<String, Double>> shares = new ArrayList<>(root.dagShare().entrySet());
shares.sort(Map.Entry.<String, Double>comparingByValue().reversed());
StringBuilder line = new StringBuilder(String.format(Locale.ROOT, " dag %-18s", root.root()));
for (int index = 0; index < Math.min(3, shares.size()); index++) {
line.append(String.format(Locale.ROOT, " %4.1f%% %s;", 100.0 * shares.get(index).getValue(), shares.get(index).getKey()));
}
out.append(line).append('\n');
}
DensityJsonBreakdown.Report json = report.densityJson();
out.append(String.format(Locale.ROOT, "density function JSON %.0f MB in %d files, %d cellular instances in %d shapes%n",
json.bytes() / MEGABYTE, json.files(), json.cellularInstances(), json.cellularShapes().size()));
for (int index = 0; index < Math.min(9, json.categories().size()); index++) {
DensityJsonBreakdown.CategoryBytes category = json.categories().get(index);
out.append(String.format(Locale.ROOT, " %-22s %8.1f MB %7d files %7d shapes %8.1f MB dup%n", category.category(),
category.bytes() / MEGABYTE, category.files(), category.distinctShapes(), category.structurallyDuplicateBytes() / MEGABYTE));
}
for (int index = 0; index < Math.min(4, json.libraryOwners().size()); index++) {
DensityJsonBreakdown.Bytes owner = json.libraryOwners().get(index);
out.append(String.format(Locale.ROOT, " %-48s %8.1f MB%n", owner.name(), owner.bytes() / MEGABYTE));
}
return out.toString();
}
private static void row(StringBuilder out, String label, double vanilla, double export) {
out.append(String.format(Locale.ROOT, "%-34s %10.3f %10.3f %8s%n", label, vanilla, export, ratio(vanilla, export)));
}
private static void row(StringBuilder out, String label, double vanilla, double export, double vanillaCpu, double exportCpu) {
out.append(String.format(Locale.ROOT, "%-34s %10.3f %10.3f %8s %10.3f %10.3f %8s%n", label, vanilla, export, ratio(vanilla, export),
vanillaCpu, exportCpu, ratio(vanillaCpu, exportCpu)));
}
private static void stage(StringBuilder out, String label, WorldgenStageBench.StageTiming vanilla, WorldgenStageBench.StageTiming export) {
row(out, label, vanilla.wall().mean(), export.wall().mean(), vanilla.cpu().mean(), export.cpu().mean());
}
private static void call(StringBuilder out, String label, WorldgenStageBench.CallTiming vanilla, WorldgenStageBench.CallTiming export) {
row(out, label, vanilla.wall().mean(), export.wall().mean(), vanilla.cpuMean(), export.cpuMean());
}
private static String ratio(double vanilla, double export) {
return vanilla > 0.0 ? String.format(Locale.ROOT, "%.1fx", export / vanilla) : "-";
}
private static DensityCostAnalysis.RootCost find(List<DensityCostAnalysis.RootCost> roots, String name) {
for (DensityCostAnalysis.RootCost root : roots) {
if (root.root().equals(name)) {
return root;
}
}
return null;
}
private static String shortName(String name) {
int separator = name.indexOf(':');
String path = separator < 0 ? name : name.substring(separator + 1);
return path.length() <= 34 ? path : "..." + path.substring(path.length() - 31);
}
private static String compact(double value) {
if (value < 1.0e4) {
return String.format(Locale.ROOT, "%.0f", value);
}
return String.format(Locale.ROOT, "%.2e", value);
}
private static void log(String message) {
System.out.println(LOG_PREFIX + " " + message);
}
private static void deleteTree(Path root) throws IOException {
if (!Files.exists(root)) {
return;
}
HeadlessNativeRegistries.delete(root);
}
record Configuration(Path pack, String dimension, long seed, int side, int rounds, int warmup, int calls, long callSeconds,
Path output, Path work) {
Configuration {
if (side < 1 || rounds < 1 || warmup < 1 || calls < MINIMUM_CALLS || callSeconds < 1) {
throw new IllegalArgumentException("side, rounds, warmup and callSeconds must be positive and calls at least " + MINIMUM_CALLS);
}
}
static Configuration parse(String[] arguments) {
if (arguments.length != 10) {
throw new IllegalArgumentException("Usage: pack dimension seed side rounds warmupSide calls callSeconds output work");
}
return new Configuration(new File(arguments[0]).toPath().toAbsolutePath(), arguments[1], Long.parseLong(arguments[2]),
Integer.parseInt(arguments[3]), Integer.parseInt(arguments[4]), Integer.parseInt(arguments[5]),
Integer.parseInt(arguments[6]), Long.parseLong(arguments[7]), new File(arguments[8]).toPath().toAbsolutePath(),
new File(arguments[9]).toPath().toAbsolutePath());
}
}
record ExportStats(double seconds, Map<String, Double> stageSeconds, long zipBytes, int files, int densityFunctions,
long densityFunctionBytes, int noises, int biomes, int diagnostics) {
}
record GeneratorStats(double registryLoadSeconds, long retainedHeapBytes, long samplerHeapBytes, WorldgenStageBench.ChunkTimings chunks,
WorldgenStageBench.CallTiming uncachedBiome, WorldgenStageBench.CallTiming baseHeightSurface,
WorldgenStageBench.CallTiming baseHeightFloor, WorldgenStageBench.CallTiming baseColumn) {
}
record MachineLoad(int processors, double loadAverageStart, double loadAverageEnd) {
}
record Report(String pack, String dimension, long seed, int chunks, int warmupChunks, int rounds, MachineLoad load,
String threading, String decoration,
ExportStats export, Map<String, GeneratorStats> generators, DensityJsonBreakdown.Report densityJson,
Map<String, List<DensityCostAnalysis.RootCost>> roots, Map<String, String> definitions) {
}
private record ExportRun(ExportStats stats, SortedMap<String, byte[]> files) {
}
private static final class Measurement {
private final WorldgenStageBench bench;
private final WorldgenStageBench.ChunkSamples chunks = new WorldgenStageBench.ChunkSamples();
private final WorldgenStageBench.CallSamples uncachedBiome = new WorldgenStageBench.CallSamples();
private final WorldgenStageBench.CallSamples surfaceHeight = new WorldgenStageBench.CallSamples();
private final WorldgenStageBench.CallSamples floorHeight = new WorldgenStageBench.CallSamples();
private final WorldgenStageBench.CallSamples column = new WorldgenStageBench.CallSamples();
private Measurement(WorldgenStageBench bench) {
this.bench = bench;
}
private GeneratorStats stats(double loadSeconds, long retainedHeap, long samplerHeap) {
return new GeneratorStats(loadSeconds, retainedHeap, samplerHeap, chunks.timings(), uncachedBiome.timing(),
surfaceHeight.timing(), floorHeight.timing(), column.timing());
}
}
private static final class StageClock implements WorldgenExportProgress {
private final Map<String, Long> starts = new LinkedHashMap<>();
private long origin;
private void start(long nanos) {
origin = nanos;
}
@Override
public void stage(String stage) {
starts.put(stage, System.nanoTime());
}
@Override
public void resources(int done, int total) {
}
private Map<String, Double> seconds(long finished) {
Map<String, Double> seconds = new LinkedHashMap<>();
String previous = "setup";
long previousStart = origin;
for (Map.Entry<String, Long> entry : starts.entrySet()) {
seconds.put(previous, (entry.getValue() - previousStart) / NANOS_PER_SECOND);
previous = entry.getKey();
previousStart = entry.getValue();
}
seconds.put(previous, (finished - previousStart) / NANOS_PER_SECOND);
return seconds;
}
}
}
@@ -0,0 +1,331 @@
package art.arcane.iris.probe;
import art.arcane.iris.generation.concurrent.MultiBurst;
import art.arcane.iris.generation.runtime.Engine;
import art.arcane.iris.pack.datapack.export.VanillaResourceIdentifier;
import art.arcane.iris.pack.datapack.export.WorldgenDatapackExporter;
import art.arcane.iris.pack.datapack.export.WorldgenExportMode;
import art.arcane.iris.pack.datapack.export.WorldgenExportProgress;
import art.arcane.iris.pack.datapack.export.WorldgenExportRequest;
import art.arcane.iris.pack.datapack.export.WorldgenExportResult;
import art.arcane.iris.pack.datapack.export.WorldgenExportService;
import art.arcane.iris.pack.datapack.export.compile.surface.MaterialProgramCompiler;
import art.arcane.iris.pack.datapack.export.ir.ExportDiagnostic;
import art.arcane.iris.probe.fidelity.BiomeShareReport;
import art.arcane.iris.probe.fidelity.BiomeTypes;
import art.arcane.iris.probe.fidelity.ColumnSource;
import art.arcane.iris.probe.fidelity.ColumnSurvey;
import art.arcane.iris.probe.fidelity.FidelityComparison;
import art.arcane.iris.probe.fidelity.FidelityReport;
import art.arcane.iris.probe.fidelity.FidelitySampling;
import art.arcane.iris.probe.fidelity.FidelitySamples;
import art.arcane.iris.probe.fidelity.FidelityScanner;
import art.arcane.iris.probe.fidelity.FidelitySource;
import art.arcane.iris.probe.fidelity.FidelityStats;
import art.arcane.iris.probe.fidelity.FidelityThresholds;
import art.arcane.iris.probe.fidelity.IrisEngineColumnSource;
import art.arcane.iris.probe.fidelity.IrisEngineFidelitySource;
import art.arcane.iris.probe.fidelity.SurveyWindow;
import art.arcane.iris.probe.fidelity.VanillaNoiseColumnSource;
import art.arcane.iris.probe.fidelity.VanillaNoiseFidelitySource;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.StandardCopyOption;
import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Objects;
import java.util.SortedMap;
import java.util.TreeMap;
public final class DatapackFidelityProbe {
private static final String LOG_PREFIX = "[datapack-fidelity]";
private static final String USAGE = "Usage: datapackFidelityProbe pack dimension seed radiusChunks stride output surveyBlocks surveyStride surveySeeds mode";
private static final String TARGET_VERSION = "26.3";
private static final String EXPORT_FORMAT = "directory";
private static final String EXPORT_OUTPUT = "export";
private static final int MAX_STRIDE = 16;
private static final int CHUNK_WIDTH = 16;
private static final int ARGUMENTS = 10;
private static final String COLUMN_SCOPE = "columns";
private static final String CHUNK_SCOPE = "chunks";
private static final int EXIT_ERROR = 1;
private static final int EXIT_FIDELITY_FAILED = 3;
private DatapackFidelityProbe() {
}
public static void main(String[] arguments) {
int status;
try {
status = execute(arguments, WorldgenDatapackExporter.create()).passed() ? 0 : EXIT_FIDELITY_FAILED;
} catch (Throwable failure) {
failure.printStackTrace(System.err);
status = EXIT_ERROR;
}
System.out.flush();
System.err.flush();
System.exit(status);
}
static FidelityReport execute(String[] arguments, WorldgenExportService exporter) throws Exception {
Objects.requireNonNull(exporter, "exporter");
Configuration configuration = Configuration.parse(arguments);
System.out.println(LOG_PREFIX + " pack=" + configuration.pack() + " dimension=" + configuration.dimension()
+ " seed=" + configuration.seed() + " radiusChunks=" + configuration.radiusChunks() + " stride=" + configuration.stride()
+ " surveyBlocks=" + configuration.survey().size() + " surveyStride=" + configuration.survey().stride()
+ " surveySeeds=" + configuration.surveySeeds() + " mode=" + configuration.mode().label());
Outcome outcome = compare(configuration, exporter);
writeReport(configuration.output(), outcome.json());
System.out.print(outcome.table());
System.out.println(LOG_PREFIX + " report=" + configuration.output());
return outcome.report();
}
private static Outcome compare(Configuration configuration, WorldgenExportService exporter) throws Exception {
Path scratch = Files.createTempDirectory("iris-datapack-fidelity-");
try {
return compareIn(scratch, configuration, exporter);
} finally {
ProbeWorkspaceCleanup.deleteRecursively(scratch);
}
}
private static Outcome compareIn(Path scratch, Configuration configuration, WorldgenExportService exporter) throws Exception {
HeadlessNativeRuntime runtime = new HeadlessNativeRuntime();
try (RealPackProbeSupport.Workspace workspace = RealPackProbeSupport.openWorkspace(new RealPackProbeSupport.WorkspaceOptions(
configuration.pack().toFile(), configuration.dimension(), LOG_PREFIX, runtime, scratch))) {
WorldgenExportResult export = export(exporter, workspace.pack().toPath(), configuration.dimension(), scratch.resolve("exports"));
Target target = new Target(export.output(), WorldgenExportRequest.DEFAULT_NAMESPACE,
VanillaResourceIdentifier.sanitizePath(configuration.dimension()), export.loadKeyByBiomeId());
return switch (configuration.mode()) {
case FULL -> Outcome.of(compareAll(workspace, target, configuration));
case BIOMES -> Outcome.of(compareBiomes(workspace, target, configuration));
};
}
}
private static FidelityReport compareAll(RealPackProbeSupport.Workspace workspace, Target target, Configuration configuration) throws Exception {
List<Long> seeds = configuration.surveySeeds();
FidelityStats irisColumns = null;
FidelityStats exportColumns = null;
FidelitySamples chunks = null;
for (long seed : seeds) {
try (RealPackProbeSupport.EngineSession session = workspace.openEngine(seed, false, "fidelity-" + seed)) {
Surveyed surveyed = survey(session.engine(), target, seed, configuration.survey());
irisColumns = irisColumns == null ? surveyed.iris() : irisColumns.merge(surveyed.iris());
exportColumns = exportColumns == null ? surveyed.export() : exportColumns.merge(surveyed.export());
if (seed == configuration.seed()) {
chunks = scanChunks(session.engine(), target, configuration);
}
}
}
FidelitySampling columnSampling = new FidelitySampling(COLUMN_SCOPE, configuration.survey().size(), configuration.survey().stride(),
seeds, irisColumns.columns());
return FidelityComparison.compare(new FidelitySamples(irisColumns, exportColumns, columnSampling), chunks, FidelityThresholds.defaults());
}
private static BiomeShareReport compareBiomes(RealPackProbeSupport.Workspace workspace, Target target, Configuration configuration) throws Exception {
List<FidelitySamples> seeds = new ArrayList<>(configuration.seedCount());
Map<String, String> biomeTypes = Map.of();
for (long seed : configuration.surveySeeds()) {
try (RealPackProbeSupport.EngineSession session = workspace.openEngine(seed, false, "biomes-" + seed)) {
Engine engine = session.engine();
Surveyed surveyed = survey(engine, target, seed, configuration.survey());
if (biomeTypes.isEmpty()) {
biomeTypes = BiomeTypes.of(engine.getDimension().getAllRegions(engine), key -> engine.getData().getBiomeLoader().load(key));
}
seeds.add(new FidelitySamples(surveyed.iris(), surveyed.export(), new FidelitySampling(COLUMN_SCOPE, configuration.survey().size(),
configuration.survey().stride(), List.of(seed), surveyed.iris().columns())));
}
}
return FidelityComparison.compareBiomeShares(seeds, biomeTypes, FidelityThresholds.defaults());
}
private static Surveyed survey(Engine engine, Target target, long seed, SurveyWindow window) throws Exception {
try (ColumnSource iris = new IrisEngineColumnSource(engine);
ColumnSource vanilla = new VanillaNoiseColumnSource(target.sampler(seed), target.loadKeyByBiomeId())) {
requireSameHeightRange(iris.minY(), iris.maxY(), vanilla.minY(), vanilla.maxY());
return new Surveyed(surveyed(iris, window, seed), surveyed(vanilla, window, seed));
}
}
private static FidelityStats surveyed(ColumnSource source, SurveyWindow window, long seed) {
long started = System.nanoTime();
FidelityStats stats = ColumnSurvey.scan(source, window, MultiBurst.burst.parallelism());
System.out.println(LOG_PREFIX + " surveyed " + source.label() + " seed=" + seed + " columns=" + stats.columns()
+ " in " + (System.nanoTime() - started) / 1_000_000L + "ms");
return stats;
}
private static FidelitySamples scanChunks(Engine engine, Target target, Configuration configuration) throws Exception {
try (IrisEngineFidelitySource iris = new IrisEngineFidelitySource(engine);
VanillaNoiseFidelitySource vanilla = new VanillaNoiseFidelitySource(target.sampler(configuration.seed()), target.loadKeyByBiomeId(),
target.namespace() + ":" + MaterialProgramCompiler.WALL_SHELL)) {
requireSameHeightRange(iris.minY(), iris.maxY(), vanilla.minY(), vanilla.maxY());
FidelityStats irisStats = scan(iris, configuration);
FidelityStats exportStats = scan(vanilla, configuration);
FidelitySampling sampling = new FidelitySampling(CHUNK_SCOPE, (2 * configuration.radiusChunks() + 1) * CHUNK_WIDTH,
configuration.stride(), List.of(configuration.seed()), irisStats.columns());
return new FidelitySamples(irisStats, exportStats, sampling);
}
}
private static WorldgenExportResult export(WorldgenExportService exporter, Path pack, String dimension, Path exportRoot) throws IOException {
Files.createDirectories(exportRoot);
WorldgenExportRequest request = WorldgenExportRequest.builder()
.pack(pack)
.dimension(dimension)
.version(TARGET_VERSION)
.format(EXPORT_FORMAT)
.output(EXPORT_OUTPUT)
.exportRoot(exportRoot)
.build();
WorldgenExportResult result = exporter.export(request, WorldgenExportProgress.NONE);
if (!result.isSuccess()) {
throw new IllegalStateException("Datapack export of dimension '" + dimension + "' was " + result.status() + ": " + result.message());
}
System.out.println(LOG_PREFIX + " exported target=" + TARGET_VERSION + " biomes=" + result.loadKeyByBiomeId().size()
+ " diagnostics=" + result.diagnostics().size() + " " + diagnosticCounts(result.diagnostics()));
return result;
}
private static FidelityStats scan(FidelitySource source, Configuration configuration) {
long started = System.nanoTime();
FidelityStats stats = FidelityScanner.scan(source, 0, 0, configuration.radiusChunks(), configuration.stride(),
MultiBurst.burst.parallelism());
System.out.println(LOG_PREFIX + " sampled " + source.label() + " columns=" + stats.columns() + " terrainColumns=" + stats.heightColumns()
+ " in " + (System.nanoTime() - started) / 1_000_000L + "ms");
return stats;
}
private static void requireSameHeightRange(int irisMinY, int irisMaxY, int exportMinY, int exportMaxY) {
if (irisMinY != exportMinY || irisMaxY != exportMaxY) {
throw new IllegalStateException("Iris height range [" + irisMinY + ", " + irisMaxY
+ ") differs from the exported dimension range [" + exportMinY + ", " + exportMaxY + ")");
}
}
private static SortedMap<String, Integer> diagnosticCounts(List<ExportDiagnostic> diagnostics) {
SortedMap<String, Integer> counts = new TreeMap<>();
for (ExportDiagnostic diagnostic : diagnostics) {
counts.merge(diagnostic.limit().name(), 1, Integer::sum);
}
return counts;
}
private static void writeReport(Path output, String json) throws IOException {
Path parent = output.getParent();
Files.createDirectories(parent);
Path staged = Files.createTempFile(parent, ".datapack-fidelity-", ".json");
try {
Files.writeString(staged, json + "\n", StandardCharsets.UTF_8);
Files.move(staged, output, StandardCopyOption.REPLACE_EXISTING, StandardCopyOption.ATOMIC_MOVE);
} finally {
Files.deleteIfExists(staged);
}
}
private enum Mode {
FULL,
BIOMES;
private static Mode parse(String value) {
String normalized = value.trim().toLowerCase(Locale.ROOT);
for (Mode mode : values()) {
if (mode.label().equals(normalized)) {
return mode;
}
}
throw new IllegalArgumentException("mode must be full or biomes, got '" + value + "'");
}
private String label() {
return name().toLowerCase(Locale.ROOT);
}
}
private record Outcome(FidelityReport report, String json, String table) {
private static Outcome of(FidelityReport report) {
return new Outcome(report, report.toJson(), report.toTable());
}
private static Outcome of(BiomeShareReport report) {
return new Outcome(report.mean(), report.toJson(), report.toTable());
}
}
private record Surveyed(FidelityStats iris, FidelityStats export) {
}
private record Target(Path output, String namespace, String dimension, Map<String, String> loadKeyByBiomeId) {
private WorldgenExportSampler sampler(long seed) throws IOException {
return WorldgenExportSampler.open(output, WorldgenExportMode.OVERWORLD, namespace + ":" + dimension, seed);
}
}
private record Configuration(Path pack, String dimension, long seed, int radiusChunks, int stride, Path output, SurveyWindow survey,
int seedCount, Mode mode) {
private static Configuration parse(String[] arguments) {
if (arguments.length != ARGUMENTS) {
throw new IllegalArgumentException(USAGE);
}
Path pack = Path.of(arguments[0]).toAbsolutePath().normalize();
if (!Files.isDirectory(pack)) {
throw new IllegalArgumentException("Pack folder " + pack + " does not exist");
}
String dimension = arguments[1].trim();
if (dimension.isEmpty()) {
throw new IllegalArgumentException("Dimension key must not be blank");
}
long seed = parseLong(arguments[2], "seed");
int radiusChunks = parseInt(arguments[3], "radiusChunks");
if (radiusChunks < 0) {
throw new IllegalArgumentException("radiusChunks must be zero or positive, got " + radiusChunks);
}
int stride = parseInt(arguments[4], "stride");
if (stride < 1 || stride > MAX_STRIDE) {
throw new IllegalArgumentException("stride must be within [1, " + MAX_STRIDE + "], got " + stride);
}
Path output = Path.of(arguments[5]).toAbsolutePath().normalize();
if (Files.isDirectory(output)) {
throw new IllegalArgumentException("Report output " + output + " is a directory");
}
SurveyWindow survey = new SurveyWindow(0, 0, parseInt(arguments[6], "surveyBlocks"), parseInt(arguments[7], "surveyStride"));
int seedCount = parseInt(arguments[8], "surveySeeds");
if (seedCount < 1) {
throw new IllegalArgumentException("surveySeeds must be at least 1, got " + seedCount);
}
Mode mode = Mode.parse(arguments[9]);
return new Configuration(pack, dimension, seed, radiusChunks, stride, output, survey, seedCount, mode);
}
private List<Long> surveySeeds() {
List<Long> seeds = new ArrayList<>(seedCount);
for (int index = 0; index < seedCount; index++) {
seeds.add(seed + index);
}
return seeds;
}
private static long parseLong(String value, String name) {
try {
return Long.parseLong(value.trim());
} catch (NumberFormatException failure) {
throw new IllegalArgumentException(name + " must be an integer, got '" + value + "'", failure);
}
}
private static int parseInt(String value, String name) {
try {
return Integer.parseInt(value.trim());
} catch (NumberFormatException failure) {
throw new IllegalArgumentException(name + " must be an integer, got '" + value + "'", failure);
}
}
}
}
@@ -0,0 +1,349 @@
package art.arcane.iris.probe;
import net.minecraft.core.Holder;
import net.minecraft.util.CubicSpline;
import net.minecraft.world.level.levelgen.densityfunction.DensityFunction;
import net.minecraft.world.level.levelgen.densityfunction.DensityFunctions;
import net.minecraft.world.level.levelgen.densityfunction.generator.ConstantFunction;
import net.minecraft.world.level.levelgen.densityfunction.op.CacheFunction;
import net.minecraft.world.level.levelgen.densityfunction.op.FindTopSurfaceFunction;
import net.minecraft.world.level.levelgen.densityfunction.op.InterpolatedFunction;
import net.minecraft.world.level.levelgen.densityfunction.op.IntervalSelectFunction;
import net.minecraft.world.level.levelgen.densityfunction.op.RangeChoiceFunction;
import net.minecraft.world.level.levelgen.densityfunction.op.SplineFunction;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.IdentityHashMap;
import java.util.List;
import java.util.Map;
import java.util.TreeMap;
import java.util.function.Function;
final class DensityCostAnalysis {
private static final double LATTICE_CORNERS = 8.0;
private static final int MAX_SURFACE_STEPS = 4096;
private static final String INLINE = "<inline>";
private final Function<String, String> categories;
private final int topCount;
private final Map<Object, Costs> costs = new IdentityHashMap<>();
private final Map<DensityFunction, Body> bodies = new IdentityHashMap<>();
private final Map<DensityFunction, String> names = new IdentityHashMap<>();
DensityCostAnalysis(Function<String, String> categories, int topCount) {
this.categories = categories;
this.topCount = topCount;
}
RootCost analyze(String root, DensityFunction function) {
Costs total = cost(function);
Reach reach = new Reach();
reach(function, reach, categories.apply(rootName(function, root)));
double cached = total.cachedLocal();
for (CacheFunction cache : reach.caches.keySet()) {
cached += cost(cache.input()).cachedLocal();
}
Body rootBody = new Body();
local(function, rootBody, true);
Map<DensityFunction, Double> multiplicity = multiplicity(rootBody);
List<Offender> offenders = new ArrayList<>(multiplicity.size());
Map<String, Double> shares = new TreeMap<>();
double expanded = total.expanded();
shares.merge(categories.apply(rootName(function, root)), rootBody.self / expanded, Double::sum);
for (Map.Entry<DensityFunction, Double> entry : multiplicity.entrySet()) {
DensityFunction value = entry.getKey();
Body body = body(value);
String name = names.get(value);
String category = categories.apply(name);
double contribution = entry.getValue() * body.self;
shares.merge(category, contribution / expanded, Double::sum);
offenders.add(new Offender(name, category, entry.getValue(), body.self, cost(value).expanded(),
contribution, entry.getValue() * cost(value).expanded()));
}
List<Offender> exclusive = top(offenders, Comparator.comparingDouble(Offender::contribution).reversed());
List<Offender> inclusive = top(offenders, Comparator.comparingDouble(Offender::inclusive).reversed());
Map<String, Double> dagShares = new TreeMap<>();
for (Map.Entry<String, Double> entry : reach.categories.entrySet()) {
dagShares.put(entry.getKey(), entry.getValue() / Math.max(1L, reach.nodes));
}
return new RootCost(root, reach.nodes, cached, expanded, total.lazy(), multiplicity.size(), exclusive, inclusive, shares, dagShares);
}
private List<Offender> top(List<Offender> offenders, Comparator<Offender> order) {
List<Offender> sorted = new ArrayList<>(offenders);
sorted.sort(order);
return List.copyOf(sorted.subList(0, Math.min(topCount, sorted.size())));
}
private String rootName(DensityFunction function, String root) {
if (function instanceof DensityFunctions.HolderHolder(Holder<DensityFunction> holder)) {
return name(holder);
}
return root;
}
private Costs cost(DensityFunction function) {
if (function instanceof DensityFunctions.HolderHolder(Holder<DensityFunction> holder)) {
DensityFunction value = holder.value();
names.putIfAbsent(value, name(holder));
return cost(value);
}
Costs known = costs.get(function);
if (known != null) {
return known;
}
Costs computed = compute(function);
costs.put(function, computed);
return computed;
}
private Costs compute(DensityFunction function) {
return switch (function) {
case ConstantFunction ignored -> new Costs(0.0, 0.0, 0.0);
case CacheFunction cache -> {
Costs input = cost(cache.input());
yield new Costs(1.0 + input.expanded(), 1.0 + input.lazy(), 1.0);
}
case SplineFunction spline -> {
Costs inner = spline(spline.spline());
yield new Costs(1.0 + inner.expanded(), 1.0 + inner.lazy(), 1.0 + inner.cachedLocal());
}
case IntervalSelectFunction select -> {
Costs input = cost(select.input());
double expanded = 1.0 + input.expanded();
double cached = 1.0 + input.cachedLocal();
double lazy = 0.0;
for (DensityFunction branch : select.functions()) {
Costs branchCost = cost(branch);
expanded += branchCost.expanded();
cached += branchCost.cachedLocal();
lazy = Math.max(lazy, branchCost.lazy());
}
yield new Costs(expanded, 1.0 + input.lazy() + lazy, cached);
}
case RangeChoiceFunction choice -> {
Costs input = cost(choice.input());
Costs inside = cost(choice.whenInRange());
Costs outside = cost(choice.whenOutOfRange());
yield new Costs(1.0 + input.expanded() + inside.expanded() + outside.expanded(),
1.0 + input.lazy() + Math.max(inside.lazy(), outside.lazy()),
1.0 + input.cachedLocal() + inside.cachedLocal() + outside.cachedLocal());
}
case InterpolatedFunction grid -> {
Costs input = cost(grid.input());
yield new Costs(1.0 + input.expanded(), 1.0 + LATTICE_CORNERS * input.expanded(), 1.0 + input.cachedLocal());
}
case FindTopSurfaceFunction surface -> {
Costs density = cost(surface.density());
Costs upper = cost(surface.upperBound());
double steps = surfaceSteps(surface);
yield new Costs(1.0 + density.expanded() + upper.expanded(), 1.0 + upper.lazy() + steps * density.lazy(),
1.0 + density.cachedLocal() + upper.cachedLocal());
}
default -> {
double expanded = 1.0;
double lazy = 1.0;
double cached = 1.0;
for (DensityFunction child : children(function)) {
Costs childCost = cost(child);
expanded += childCost.expanded();
lazy += childCost.lazy();
cached += childCost.cachedLocal();
}
yield new Costs(expanded, lazy, cached);
}
};
}
private Costs spline(CubicSpline<SplineFunction.Coordinate> spline) {
if (!(spline instanceof CubicSpline.Multipoint<SplineFunction.Coordinate> multipoint)) {
return new Costs(0.0, 0.0, 0.0);
}
Costs known = costs.get(multipoint);
if (known != null) {
return known;
}
Costs coordinate = cost(multipoint.coordinate().function());
List<CubicSpline<SplineFunction.Coordinate>> values = multipoint.values();
double expanded = 1.0 + coordinate.expanded();
double cached = 1.0 + coordinate.cachedLocal();
double[] lazy = new double[values.size()];
for (int index = 0; index < values.size(); index++) {
Costs value = spline(values.get(index));
expanded += value.expanded();
cached += value.cachedLocal();
lazy[index] = value.lazy();
}
double worstPair = lazy.length == 1 ? lazy[0] : 0.0;
for (int index = 0; index + 1 < lazy.length; index++) {
worstPair = Math.max(worstPair, lazy[index] + lazy[index + 1]);
}
Costs computed = new Costs(expanded, 1.0 + coordinate.lazy() + worstPair, cached);
costs.put(multipoint, computed);
return computed;
}
private void reach(DensityFunction function, Reach reach, String category) {
if (function instanceof DensityFunctions.HolderHolder(Holder<DensityFunction> holder)) {
reach(holder.value(), reach, categories.apply(name(holder)));
return;
}
if (function instanceof ConstantFunction || reach.visited.put(function, Boolean.TRUE) != null) {
return;
}
reach.count(category);
if (function instanceof CacheFunction cache) {
reach.caches.put(cache, Boolean.TRUE);
}
if (function instanceof SplineFunction spline) {
reachSpline(spline.spline(), reach, category);
return;
}
for (DensityFunction child : children(function)) {
reach(child, reach, category);
}
}
private void reachSpline(CubicSpline<SplineFunction.Coordinate> spline, Reach reach, String category) {
if (!(spline instanceof CubicSpline.Multipoint<SplineFunction.Coordinate> multipoint)) {
return;
}
if (reach.visited.put(multipoint, Boolean.TRUE) != null) {
return;
}
reach.count(category);
reach(multipoint.coordinate().function(), reach, category);
for (CubicSpline<SplineFunction.Coordinate> value : multipoint.values()) {
reachSpline(value, reach, category);
}
}
private Body body(DensityFunction value) {
Body known = bodies.get(value);
if (known != null) {
return known;
}
Body body = new Body();
local(value, body, true);
bodies.put(value, body);
return body;
}
private void local(DensityFunction function, Body body, boolean top) {
if (function instanceof DensityFunctions.HolderHolder(Holder<DensityFunction> holder)) {
if (!top) {
DensityFunction value = holder.value();
names.putIfAbsent(value, name(holder));
body.references.merge(value, 1.0, Double::sum);
return;
}
local(holder.value(), body, true);
return;
}
if (function instanceof ConstantFunction) {
return;
}
body.self++;
if (function instanceof SplineFunction spline) {
localSpline(spline.spline(), body);
return;
}
for (DensityFunction child : children(function)) {
local(child, body, false);
}
}
private void localSpline(CubicSpline<SplineFunction.Coordinate> spline, Body body) {
if (!(spline instanceof CubicSpline.Multipoint<SplineFunction.Coordinate> multipoint)) {
return;
}
body.self++;
local(multipoint.coordinate().function(), body, false);
for (CubicSpline<SplineFunction.Coordinate> value : multipoint.values()) {
localSpline(value, body);
}
}
private Map<DensityFunction, Double> multiplicity(Body root) {
List<DensityFunction> order = new ArrayList<>();
Map<DensityFunction, Boolean> visited = new IdentityHashMap<>();
for (DensityFunction child : root.references.keySet()) {
postOrder(child, visited, order);
}
Collections.reverse(order);
Map<DensityFunction, Double> multiplicity = new IdentityHashMap<>(order.size() * 2);
for (Map.Entry<DensityFunction, Double> entry : root.references.entrySet()) {
multiplicity.merge(entry.getKey(), entry.getValue(), Double::sum);
}
for (DensityFunction value : order) {
double times = multiplicity.getOrDefault(value, 0.0);
for (Map.Entry<DensityFunction, Double> entry : body(value).references.entrySet()) {
multiplicity.merge(entry.getKey(), times * entry.getValue(), Double::sum);
}
}
return multiplicity;
}
private void postOrder(DensityFunction value, Map<DensityFunction, Boolean> visited, List<DensityFunction> order) {
if (visited.put(value, Boolean.TRUE) != null) {
return;
}
for (DensityFunction child : body(value).references.keySet()) {
postOrder(child, visited, order);
}
order.add(value);
}
private static double surfaceSteps(FindTopSurfaceFunction surface) {
double upper = surface.upperBound().range().max();
if (!Double.isFinite(upper)) {
return MAX_SURFACE_STEPS;
}
double steps = Math.floor((upper - surface.lowerBound()) / surface.cellHeight()) + 1.0;
return Math.clamp(steps, 1.0, MAX_SURFACE_STEPS);
}
private static List<DensityFunction> children(DensityFunction function) {
List<DensityFunction> children = new ArrayList<>(4);
function.rewriteChildren(child -> {
children.add(child);
return child;
});
return children;
}
private static String name(Holder<DensityFunction> holder) {
return holder.unwrapKey().map(key -> key.identifier().toString()).orElse(INLINE);
}
record Costs(double expanded, double lazy, double cachedLocal) {
}
record Offender(String name, String category, double multiplicity, double selfNodes, double subtreeNodes,
double contribution, double inclusive) {
}
record RootCost(String root, long dagNodes, double cachedNodes, double expandedNodes, double uncachedPointWorst,
int namedSubtrees, List<Offender> exclusive, List<Offender> inclusive, Map<String, Double> categoryShare,
Map<String, Double> dagShare) {
}
private static final class Reach {
private final Map<Object, Boolean> visited = new IdentityHashMap<>();
private final Map<CacheFunction, Boolean> caches = new IdentityHashMap<>();
private final Map<String, Double> categories = new TreeMap<>();
private long nodes;
private void count(String category) {
nodes++;
categories.merge(category, 1.0, Double::sum);
}
}
private static final class Body {
private final Map<DensityFunction, Double> references = new IdentityHashMap<>();
private double self;
}
}
@@ -0,0 +1,409 @@
package art.arcane.iris.probe;
import com.google.gson.JsonArray;
import com.google.gson.JsonElement;
import com.google.gson.JsonObject;
import com.google.gson.JsonParser;
import com.google.gson.JsonPrimitive;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.SortedMap;
import java.util.TreeMap;
final class DensityJsonBreakdown {
private static final String DENSITY_FUNCTIONS = "/worldgen/density_function/";
private static final String NOISES = "/worldgen/noise/";
private static final String JSON = ".json";
private static final String CELLULAR = "cellular";
private static final String NOISE_STYLE = "noise style";
private static final String UNREFERENCED = "unreferenced";
private static final String OFFSET_MARKER = "/offset_";
private static final String OFFSET_SUFFIX = " offset copies";
private static final int INSTANCE_SEGMENTS = 3;
private static final Set<String> ROUTER_ROOTS = Set.of("final_density", "temperature", "vegetation", "continents", "erosion",
"depth", "ridges", "chunk_surface_level", "barrier", "fluid_level_floodedness", "fluid_level_spread", "lava", "exclusion",
"surface_level", "spawn_elevation");
private static final long FNV_OFFSET = 0xcbf29ce484222325L;
private static final long FNV_PRIME = 0x100000001b3L;
private static final int TOP = 10;
private final String dimension;
private final String dimensionPrefix;
private final Map<String, Integer> categoryIndex = new HashMap<>();
private final List<String> categoryNames = new ArrayList<>();
private final Map<String, Entry> entries = new TreeMap<>();
private final Map<String, List<String>> parents = new HashMap<>();
private final Map<String, Long> owners = new HashMap<>();
private DensityJsonBreakdown(String dimension) {
this.dimension = dimension;
dimensionPrefix = dimension + "/";
}
static DensityJsonBreakdown analyze(SortedMap<String, byte[]> files, String dimension) {
DensityJsonBreakdown breakdown = new DensityJsonBreakdown(dimension);
breakdown.load(files);
return breakdown;
}
static String category(String dimension, String id) {
String base = baseCategory(dimension, id);
return id.contains(OFFSET_MARKER) ? base + OFFSET_SUFFIX : base;
}
private static String baseCategory(String dimension, String id) {
int separator = id.indexOf(':');
String path = separator < 0 ? id : id.substring(separator + 1);
String dimensionPrefix = dimension + "/";
if (path.startsWith(dimensionPrefix)) {
path = path.substring(dimensionPrefix.length());
}
if (ROUTER_ROOTS.contains(path)) {
return "router";
}
if (path.startsWith("cell/")) {
return CELLULAR;
}
if (path.startsWith("style/")) {
return NOISE_STYLE;
}
if (path.startsWith("biome/selector/")) {
return "biome selector";
}
if (path.startsWith("biome/carving/")) {
return "biome carving";
}
if (path.startsWith("biome/")) {
return "biome decision";
}
if (path.equals("height") || path.startsWith("height/") || path.startsWith("terrain/height/")) {
return "terrain height";
}
if (path.startsWith("terrain/generator/")) {
return "terrain generator";
}
if (path.startsWith("terrain/volume/")) {
return "terrain volumetric";
}
if (path.startsWith("terrain/")) {
return "terrain bands";
}
if (path.startsWith("cave")) {
return "caves";
}
if (path.startsWith("surface/")) {
return "surface";
}
int slash = path.indexOf('/');
return slash < 0 ? path : path.substring(0, slash);
}
Report report() {
Map<String, CategoryAccumulator> categories = new TreeMap<>();
Map<String, CategoryAccumulator> groups = new TreeMap<>();
Map<String, Long> libraryOwners = new TreeMap<>();
long totalBytes = 0;
long totalNodes = 0;
for (Map.Entry<String, Entry> item : entries.entrySet()) {
String id = item.getKey();
Entry entry = item.getValue();
totalBytes += entry.bytes;
totalNodes += entry.nodes;
String category = category(dimension, id);
categories.computeIfAbsent(category, ignored -> new CategoryAccumulator()).add(entry);
groups.computeIfAbsent(group(id), ignored -> new CategoryAccumulator()).add(entry);
if (isLibrary(category)) {
libraryOwners.merge(category + " via " + ownerLabel(owners(id)), entry.bytes, Long::sum);
}
}
List<CategoryBytes> categoryRows = rows(categories, Integer.MAX_VALUE);
List<CategoryBytes> groupRows = rows(groups, 40);
List<Bytes> ownerRows = new ArrayList<>();
for (Map.Entry<String, Long> owner : libraryOwners.entrySet()) {
ownerRows.add(new Bytes(owner.getKey(), owner.getValue()));
}
ownerRows.sort(Comparator.comparingLong(Bytes::bytes).reversed());
List<Bytes> largest = new ArrayList<>();
for (Map.Entry<String, Entry> item : entries.entrySet()) {
largest.add(new Bytes(item.getKey(), item.getValue().bytes));
}
largest.sort(Comparator.comparingLong(Bytes::bytes).reversed());
return new Report(entries.size(), totalBytes, totalNodes, categoryRows, groupRows, ownerRows,
List.copyOf(largest.subList(0, Math.min(TOP, largest.size()))), cellularInstances());
}
private void load(SortedMap<String, byte[]> files) {
Map<String, byte[]> bodies = new HashMap<>();
Set<String> noises = new HashSet<>();
for (Map.Entry<String, byte[]> file : files.entrySet()) {
String id = resourceId(file.getKey(), DENSITY_FUNCTIONS);
if (id != null) {
bodies.put(id, file.getValue());
continue;
}
String noise = resourceId(file.getKey(), NOISES);
if (noise != null) {
noises.add(noise);
}
}
for (Map.Entry<String, byte[]> body : bodies.entrySet()) {
JsonElement json = JsonParser.parseString(new String(body.getValue(), StandardCharsets.UTF_8));
Entry entry = new Entry(body.getValue().length);
entry.shape = shape(json, bodies.keySet(), noises, entry);
entries.put(body.getKey(), entry);
for (String reference : entry.references) {
parents.computeIfAbsent(reference, ignored -> new ArrayList<>()).add(body.getKey());
}
}
}
private long shape(JsonElement element, Set<String> functions, Set<String> noises, Entry entry) {
if (element.isJsonObject()) {
entry.nodes++;
JsonObject object = element.getAsJsonObject();
long hash = mix(FNV_OFFSET, '{');
for (String key : new TreeMap<>(object.asMap()).keySet()) {
hash = mix(mix(hash, key.hashCode()), shape(object.get(key), functions, noises, entry));
}
return mix(hash, '}');
}
if (element.isJsonArray()) {
JsonArray array = element.getAsJsonArray();
long hash = mix(FNV_OFFSET, '[');
for (JsonElement item : array) {
hash = mix(hash, shape(item, functions, noises, entry));
}
return mix(hash, ']');
}
if (element.isJsonPrimitive()) {
JsonPrimitive primitive = element.getAsJsonPrimitive();
if (primitive.isNumber()) {
return mix(FNV_OFFSET, 'N');
}
if (primitive.isString()) {
String value = primitive.getAsString();
if (functions.contains(value)) {
entry.references.add(value);
return mix(FNV_OFFSET, 'R');
}
if (noises.contains(value)) {
return mix(FNV_OFFSET, 'Z');
}
return mix(FNV_OFFSET, value.hashCode());
}
return mix(FNV_OFFSET, primitive.getAsBoolean() ? 'T' : 'F');
}
return mix(FNV_OFFSET, '0');
}
private List<CellularShape> cellularInstances() {
Map<String, List<String>> instances = new TreeMap<>();
for (String id : entries.keySet()) {
if (!isLibrary(category(dimension, id)) || !id.contains(":cell/")) {
continue;
}
instances.computeIfAbsent(instanceKey(id), ignored -> new ArrayList<>()).add(id);
}
Map<Long, CellularAccumulator> shapes = new HashMap<>();
for (Map.Entry<String, List<String>> instance : instances.entrySet()) {
List<String> members = instance.getValue();
members.sort(String::compareTo);
long hash = FNV_OFFSET;
long bytes = 0;
long owner = 0L;
StringBuilder layout = new StringBuilder();
for (String member : members) {
Entry entry = entries.get(member);
String suffix = member.substring(instance.getKey().length());
hash = mix(mix(hash, suffix.hashCode()), entry.shape);
bytes += entry.bytes;
owner |= owners(member);
if (!layout.isEmpty()) {
layout.append(',');
}
layout.append(suffix);
}
CellularAccumulator accumulator = shapes.computeIfAbsent(hash, ignored -> new CellularAccumulator(instance.getKey(), layout.toString()));
accumulator.instances++;
accumulator.bytes += bytes;
accumulator.owners |= owner;
}
List<CellularShape> rows = new ArrayList<>(shapes.size());
for (CellularAccumulator accumulator : shapes.values()) {
rows.add(new CellularShape(accumulator.example, accumulator.layout, accumulator.instances, accumulator.bytes, ownerLabel(accumulator.owners)));
}
rows.sort(Comparator.comparingLong(CellularShape::bytes).reversed());
return List.copyOf(rows);
}
private long owners(String id) {
Long known = owners.get(id);
if (known != null) {
return known;
}
owners.put(id, 0L);
long mask = 0L;
List<String> referrers = parents.get(id);
if (referrers == null) {
mask = bit(UNREFERENCED);
} else {
for (String parent : referrers) {
String category = category(dimension, parent);
mask |= isLibrary(category) ? owners(parent) : bit(category);
}
}
owners.put(id, mask);
return mask;
}
private long bit(String category) {
int index = categoryIndex.computeIfAbsent(category, name -> {
categoryNames.add(name);
return categoryNames.size() - 1;
});
if (index >= Long.SIZE) {
throw new IllegalStateException("More than " + Long.SIZE + " density function owner categories");
}
return 1L << index;
}
private String ownerLabel(long mask) {
if (mask == 0L) {
return UNREFERENCED;
}
List<String> labels = new ArrayList<>(Long.bitCount(mask));
for (int index = 0; index < categoryNames.size(); index++) {
if ((mask & (1L << index)) != 0L) {
labels.add(categoryNames.get(index));
}
}
labels.sort(String::compareTo);
return String.join("+", labels);
}
private String group(String id) {
int separator = id.indexOf(':');
String path = separator < 0 ? id : id.substring(separator + 1);
if (path.startsWith(dimensionPrefix)) {
path = path.substring(dimensionPrefix.length());
}
String[] segments = path.split("/");
if (segments.length == 1) {
return segments[0];
}
if (segments[0].equals("cell") || segments[0].equals("style")) {
return segments[0] + "/*/" + segments[segments.length - 1];
}
return segments[0] + "/" + segments[1];
}
private static boolean isLibrary(String category) {
return category.startsWith(CELLULAR) || category.startsWith(NOISE_STYLE);
}
private static String instanceKey(String id) {
int end = id.indexOf(':') + 1;
for (int segment = 0; segment < INSTANCE_SEGMENTS && end > 0; segment++) {
int slash = id.indexOf('/', end);
if (slash < 0) {
return id;
}
end = slash + 1;
}
return id.substring(0, end - 1);
}
private static List<CategoryBytes> rows(Map<String, CategoryAccumulator> accumulators, int limit) {
List<CategoryBytes> rows = new ArrayList<>(accumulators.size());
for (Map.Entry<String, CategoryAccumulator> item : accumulators.entrySet()) {
CategoryAccumulator accumulator = item.getValue();
long representative = 0;
for (long bytes : accumulator.representativeBytes.values()) {
representative += bytes;
}
rows.add(new CategoryBytes(item.getKey(), accumulator.files, accumulator.bytes, accumulator.nodes,
accumulator.representativeBytes.size(), accumulator.bytes - representative));
}
rows.sort(Comparator.comparingLong(CategoryBytes::bytes).reversed());
return List.copyOf(rows.subList(0, Math.min(limit, rows.size())));
}
private static String resourceId(String path, String folder) {
int marker = path.indexOf(folder);
if (!path.startsWith("data/") || marker < 0 || !path.endsWith(JSON)) {
return null;
}
return path.substring("data/".length(), marker) + ":" + path.substring(marker + folder.length(), path.length() - JSON.length());
}
private static long mix(long hash, long value) {
long mixed = hash ^ value;
mixed *= FNV_PRIME;
return mixed ^ (mixed >>> 29);
}
record Report(int files, long bytes, long nodes, List<CategoryBytes> categories, List<CategoryBytes> groups,
List<Bytes> libraryOwners, List<Bytes> largest, List<CellularShape> cellularShapes) {
int cellularInstances() {
int count = 0;
for (CellularShape shape : cellularShapes) {
count += shape.instances();
}
return count;
}
}
record CategoryBytes(String category, int files, long bytes, long nodes, int distinctShapes, long structurallyDuplicateBytes) {
}
record Bytes(String name, long bytes) {
}
record CellularShape(String example, String members, int instances, long bytes, String owners) {
}
private static final class Entry {
private final long bytes;
private final List<String> references = new ArrayList<>();
private long nodes;
private long shape;
private Entry(long bytes) {
this.bytes = bytes;
}
}
private static final class CategoryAccumulator {
private final Map<Long, Long> representativeBytes = new HashMap<>();
private int files;
private long bytes;
private long nodes;
private void add(Entry entry) {
files++;
bytes += entry.bytes;
nodes += entry.nodes;
representativeBytes.putIfAbsent(entry.shape, entry.bytes);
}
}
private static final class CellularAccumulator {
private final String example;
private final String layout;
private int instances;
private long bytes;
private long owners;
private CellularAccumulator(String example, String layout) {
this.example = example;
this.layout = layout;
}
}
}
@@ -1,6 +1,7 @@
package art.arcane.iris.probe; package art.arcane.iris.probe;
import art.arcane.iris.configuration.IrisSettings; import art.arcane.iris.configuration.IrisSettings;
import art.arcane.iris.platform.ClasspathVanillaDataSource;
import art.arcane.iris.platform.bukkit.nms.BukkitGenerationRegistry; import art.arcane.iris.platform.bukkit.nms.BukkitGenerationRegistry;
import art.arcane.iris.spi.IrisPlatform; import art.arcane.iris.spi.IrisPlatform;
import art.arcane.iris.spi.PlatformGenerationRegistry; import art.arcane.iris.spi.PlatformGenerationRegistry;
@@ -9,6 +10,7 @@ import art.arcane.iris.spi.PlatformBiomeWriter;
import art.arcane.iris.spi.PlatformRegistries; import art.arcane.iris.spi.PlatformRegistries;
import art.arcane.iris.spi.PlatformScheduler; import art.arcane.iris.spi.PlatformScheduler;
import art.arcane.iris.spi.PlatformStructureHooks; import art.arcane.iris.spi.PlatformStructureHooks;
import art.arcane.iris.spi.VanillaDataSource;
import art.arcane.volmlib.nativelib.entity.NativeEntityType; import art.arcane.volmlib.nativelib.entity.NativeEntityType;
import art.arcane.volmlib.nativelib.item.NativeItem; import art.arcane.volmlib.nativelib.item.NativeItem;
import art.arcane.volmlib.nativelib.minecraft26_2.modded.ModdedBlockState; import art.arcane.volmlib.nativelib.minecraft26_2.modded.ModdedBlockState;
@@ -35,6 +37,7 @@ final class HeadlessNativePlatform implements IrisPlatform, PlatformRegistries,
private final NativeRegistryAccess nativeAccess; private final NativeRegistryAccess nativeAccess;
private final PlatformBiomeWriter biomeWriter; private final PlatformBiomeWriter biomeWriter;
private final PlatformGenerationRegistry generationRegistry; private final PlatformGenerationRegistry generationRegistry;
private final ClasspathVanillaDataSource vanillaData = new ClasspathVanillaDataSource(() -> SharedConstants.class);
private final NativeBlockResolver blocks = new NativeBlockResolver(this); private final NativeBlockResolver blocks = new NativeBlockResolver(this);
private final NativeBlockState air = blocks.getAir(); private final NativeBlockState air = blocks.getAir();
private final ConcurrentHashMap<String, NativeBlockState> states = new ConcurrentHashMap<>(); private final ConcurrentHashMap<String, NativeBlockState> states = new ConcurrentHashMap<>();
@@ -215,6 +218,11 @@ final class HeadlessNativePlatform implements IrisPlatform, PlatformRegistries,
return biomeWriter; return biomeWriter;
} }
@Override
public VanillaDataSource vanillaData() {
return vanillaData;
}
@Override @Override
public File dataFolder() { public File dataFolder() {
return services.dataFolder(); return services.dataFolder();
@@ -4,6 +4,7 @@ import art.arcane.iris.pack.datapack.IrisDatapackCompiler;
import art.arcane.iris.pack.datapack.v263.DataFixerV263; import art.arcane.iris.pack.datapack.v263.DataFixerV263;
import art.arcane.volmlib.util.collection.KList; import art.arcane.volmlib.util.collection.KList;
import net.minecraft.core.HolderLookup;
import net.minecraft.core.RegistryAccess; import net.minecraft.core.RegistryAccess;
import net.minecraft.core.registries.BuiltInRegistries; import net.minecraft.core.registries.BuiltInRegistries;
import net.minecraft.network.chat.Component; import net.minecraft.network.chat.Component;
@@ -25,6 +26,7 @@ import java.util.Map;
import java.util.Optional; import java.util.Optional;
import java.util.concurrent.ExecutorService; import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors; import java.util.concurrent.Executors;
import java.util.function.Function;
import java.util.stream.Stream; import java.util.stream.Stream;
final class HeadlessNativeRegistries implements AutoCloseable { final class HeadlessNativeRegistries implements AutoCloseable {
@@ -70,7 +72,7 @@ final class HeadlessNativeRegistries implements AutoCloseable {
} }
} }
private static HeadlessNativeRegistries load(Path directory) { static HeadlessNativeRegistries load(Path directory) {
PackLocationInfo location = new PackLocationInfo("iris-headless", Component.literal("Iris"), PackLocationInfo location = new PackLocationInfo("iris-headless", Component.literal("Iris"),
PackSource.WORLD, Optional.empty()); PackSource.WORLD, Optional.empty());
MultiPackResourceManager resources = new MultiPackResourceManager(PackType.SERVER_DATA, List.of( MultiPackResourceManager resources = new MultiPackResourceManager(PackType.SERVER_DATA, List.of(
@@ -78,12 +80,17 @@ final class HeadlessNativeRegistries implements AutoCloseable {
try { try {
RegistryAccess.Frozen builtins = RegistryAccess.fromRegistryOfRegistries(BuiltInRegistries.REGISTRY); RegistryAccess.Frozen builtins = RegistryAccess.fromRegistryOfRegistries(BuiltInRegistries.REGISTRY);
RegistryAccess.Frozen loaded; RegistryAccess.Frozen loaded;
RegistryAccess.Frozen dimensions;
try (ExecutorService executor = Executors.newFixedThreadPool(4)) { try (ExecutorService executor = Executors.newFixedThreadPool(4)) {
loaded = RegistryDataLoader.load(resources, builtins.listRegistries().toList(), loaded = RegistryDataLoader.load(resources, builtins.listRegistries().toList(),
RegistryDataLoader.WORLD_REGISTRIES, executor).join(); RegistryDataLoader.WORLD_REGISTRIES, executor).join();
List<HolderLookup.RegistryLookup<?>> dimensionContext =
Stream.concat(builtins.listRegistries(), loaded.listRegistries()).toList();
dimensions = RegistryDataLoader.load(resources, dimensionContext,
RegistryDataLoader.DIMENSION_REGISTRIES, executor).join();
} }
RegistryAccess.Frozen registries = new RegistryAccess.ImmutableRegistryAccess( RegistryAccess.Frozen registries = new RegistryAccess.ImmutableRegistryAccess(Stream.of(
Stream.concat(builtins.registries(), loaded.registries())).freeze(); builtins.registries(), loaded.registries(), dimensions.registries()).flatMap(Function.identity())).freeze();
HeadlessNativeBootstrap.bindComponents(registries); HeadlessNativeBootstrap.bindComponents(registries);
return new HeadlessNativeRegistries(directory, resources, registries); return new HeadlessNativeRegistries(directory, resources, registries);
} catch (RuntimeException failure) { } catch (RuntimeException failure) {
@@ -106,7 +113,7 @@ final class HeadlessNativeRegistries implements AutoCloseable {
delete(directory); delete(directory);
} }
private static void delete(Path directory) throws IOException { static void delete(Path directory) throws IOException {
try (Stream<Path> paths = Files.walk(directory)) { try (Stream<Path> paths = Files.walk(directory)) {
for (Path path : paths.sorted(Comparator.reverseOrder()).toList()) { for (Path path : paths.sorted(Comparator.reverseOrder()).toList()) {
Files.delete(path); Files.delete(path);
@@ -32,6 +32,7 @@ final class HeadlessNativeRuntime implements RealPackProbeSupport.RuntimeBinding
private final Path stagingRoot; private final Path stagingRoot;
private NativeRegionTerrainWriter writer; private NativeRegionTerrainWriter writer;
private HeadlessNativeRegistries nativeRegistries; private HeadlessNativeRegistries nativeRegistries;
private File preparedPack;
private HolderLookup.Provider registries; private HolderLookup.Provider registries;
HeadlessNativeRuntime() { HeadlessNativeRuntime() {
@@ -60,12 +61,16 @@ final class HeadlessNativeRuntime implements RealPackProbeSupport.RuntimeBinding
@Override @Override
public void prepare(IrisData data, IrisDimension dimension) { public void prepare(IrisData data, IrisDimension dimension) {
try { try {
if (nativeRegistries != null) { File pack = data.getDataFolder().getCanonicalFile();
if (nativeRegistries != null && !pack.equals(preparedPack)) {
throw new IllegalStateException("Headless native runtime supports one immutable pack per process"); throw new IllegalStateException("Headless native runtime supports one immutable pack per process");
} }
nativeRegistries = HeadlessNativeRegistries.compile(data.getDataFolder()); if (nativeRegistries == null) {
nativeRegistries = HeadlessNativeRegistries.compile(data.getDataFolder());
preparedPack = pack;
writer = new NativeRegionTerrainWriter(nativeRegistries.registries());
}
registries = nativeRegistries.registries(); registries = nativeRegistries.registries();
writer = new NativeRegionTerrainWriter(nativeRegistries.registries());
} catch (IOException failure) { } catch (IOException failure) {
throw new UncheckedIOException("Cannot load native generation registries", failure); throw new UncheckedIOException("Cannot load native generation registries", failure);
} }
@@ -92,6 +97,7 @@ final class HeadlessNativeRuntime implements RealPackProbeSupport.RuntimeBinding
if (nativeRegistries != null) { if (nativeRegistries != null) {
nativeRegistries.close(); nativeRegistries.close();
nativeRegistries = null; nativeRegistries = null;
preparedPack = null;
} }
} }
@@ -0,0 +1,39 @@
package art.arcane.iris.probe;
import io.papermc.paper.registry.PaperRegistryAccess;
import java.lang.reflect.Field;
import java.util.HashMap;
import java.util.Map;
final class PaperRegistryScope implements AutoCloseable {
private final Map<Object, Object> registries;
private final Map<Object, Object> previous;
private PaperRegistryScope(Map<Object, Object> registries) {
this.registries = registries;
this.previous = new HashMap<>(registries);
registries.clear();
}
static PaperRegistryScope open() {
return new PaperRegistryScope(registries());
}
@Override
public void close() {
registries.clear();
registries.putAll(previous);
}
@SuppressWarnings("unchecked")
private static Map<Object, Object> registries() {
try {
Field field = PaperRegistryAccess.class.getDeclaredField("registries");
field.setAccessible(true);
return (Map<Object, Object>) field.get(PaperRegistryAccess.instance());
} catch (ReflectiveOperationException failure) {
throw new IllegalStateException("Paper registry access has no registries map to isolate", failure);
}
}
}
@@ -0,0 +1,301 @@
package art.arcane.iris.probe;
import art.arcane.iris.pack.datapack.export.WorldgenExportMode;
import net.minecraft.core.Holder;
import net.minecraft.core.QuartPos;
import net.minecraft.core.RegistryAccess;
import net.minecraft.core.registries.Registries;
import net.minecraft.resources.Identifier;
import net.minecraft.resources.ResourceKey;
import net.minecraft.util.datafix.DataFixers;
import net.minecraft.world.level.ChunkPos;
import net.minecraft.world.level.Level;
import net.minecraft.world.level.LevelHeightAccessor;
import net.minecraft.world.level.StructureManager;
import net.minecraft.world.level.biome.Biome;
import net.minecraft.world.level.biome.BiomeManager;
import net.minecraft.world.level.biome.BiomeResolver;
import net.minecraft.world.level.chunk.ChunkAccess;
import net.minecraft.world.level.chunk.PalettedContainerFactory;
import net.minecraft.world.level.chunk.ProtoChunk;
import net.minecraft.world.level.chunk.UpgradeData;
import net.minecraft.world.level.dimension.DimensionType;
import net.minecraft.world.level.dimension.LevelStem;
import net.minecraft.world.level.levelgen.Heightmap;
import net.minecraft.world.level.levelgen.NoiseBasedChunkGenerator;
import net.minecraft.world.level.levelgen.NoiseChunk;
import net.minecraft.world.level.levelgen.NoiseGeneratorSettings;
import net.minecraft.world.level.levelgen.NoiseSettings;
import net.minecraft.world.level.levelgen.RandomState;
import net.minecraft.world.level.levelgen.WorldGenerationContext;
import net.minecraft.world.level.levelgen.blending.Blender;
import net.minecraft.world.level.levelgen.densityfunction.DensityFunction;
import net.minecraft.world.level.levelgen.densityfunction.DensityVolume;
import net.minecraft.world.level.levelgen.densityfunction.SamplerContext;
import net.minecraft.world.level.levelgen.densityfunction.ScopedDensityBuffer;
import net.minecraft.world.level.levelgen.structure.Structure;
import net.minecraft.world.level.levelgen.structure.StructureStart;
import net.minecraft.world.level.levelgen.structure.templatesystem.StructureTemplateManager;
import net.minecraft.world.level.storage.LevelStorageSource;
import java.io.IOException;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Set;
import java.util.SortedMap;
import java.util.SortedSet;
import java.util.TreeMap;
import java.util.TreeSet;
import java.util.function.Predicate;
import java.util.stream.Stream;
public final class WorldgenExportSampler implements AutoCloseable {
private static final String DENSITY_FUNCTIONS = "/worldgen/density_function/";
private static final int CHUNK_WIDTH = 16;
private static final Method CREATE_NOISE_CHUNK = generatorMethod("createNoiseChunk", ChunkAccess.class, StructureManager.class, Blender.class,
RandomState.class, NoiseSettings.class);
private static final Method DO_FILL = generatorMethod("doFill", NoiseChunk.class, ChunkAccess.class);
private final HeadlessNativeRegistries loaded;
private final NoiseBasedChunkGenerator generator;
private final RandomState randomState;
private final ThreadLocal<BiomeResolver> resolvers;
private final LevelHeightAccessor heights;
private final SortedSet<String> densityFunctionIds;
private final ThreadLocal<SamplerContext> densityContexts = ThreadLocal.withInitial(() -> SamplerContext.builder().enableCaches().build());
private final PalettedContainerFactory containers;
private final long seed;
private final long biomeZoomSeed;
private WorldgenExportSampler(HeadlessNativeRegistries loaded, LevelStem stem, long seed, SortedSet<String> densityFunctionIds) {
this.loaded = loaded;
if (!(stem.generator() instanceof NoiseBasedChunkGenerator noiseGenerator)) {
throw new IllegalStateException("Exported dimension uses " + stem.generator().getClass().getSimpleName() + ", not a noise generator");
}
this.generator = noiseGenerator;
this.randomState = RandomState.create(loaded.registries().lookupOrThrow(Registries.NOISE), seed, noiseGenerator.generatorSettings().value());
this.resolvers = ThreadLocal.withInitial(() -> noiseGenerator.getBiomeSource().createCachingResolver(randomState));
DimensionType type = stem.type().value();
this.heights = LevelHeightAccessor.create(type.minY(), type.height());
this.densityFunctionIds = densityFunctionIds;
this.containers = PalettedContainerFactory.create(loaded.registries());
this.seed = seed;
this.biomeZoomSeed = BiomeManager.obfuscateSeed(seed);
}
public static WorldgenExportSampler open(Map<String, byte[]> files, WorldgenExportMode mode, String dimensionId, long seed) throws IOException {
Objects.requireNonNull(mode);
Identifier dimension = Identifier.parse(dimensionId);
HeadlessNativeBootstrap.initialize();
Path directory = Files.createTempDirectory("iris-worldgen-export-");
HeadlessNativeRegistries loaded;
try {
for (Map.Entry<String, byte[]> file : files.entrySet()) {
Path target = directory.resolve(file.getKey()).normalize();
if (!target.startsWith(directory)) {
throw new IllegalArgumentException("Export file escapes the datapack root: " + file.getKey());
}
Files.createDirectories(target.getParent());
Files.write(target, file.getValue());
}
try (PaperRegistryScope scope = PaperRegistryScope.open()) {
loaded = HeadlessNativeRegistries.load(directory);
}
} catch (IOException | RuntimeException failure) {
HeadlessNativeRegistries.delete(directory);
throw failure;
}
try {
return new WorldgenExportSampler(loaded, stem(loaded.registries(), mode, dimension), seed, densityFunctionIds(files));
} catch (RuntimeException failure) {
loaded.close();
throw failure;
}
}
public static WorldgenExportSampler open(Path datapack, WorldgenExportMode mode, String dimensionId, long seed) throws IOException {
return open(files(datapack), mode, dimensionId, seed);
}
public float density(String functionId, int x, int y, int z) {
DensityFunction function = loaded.registries().lookupOrThrow(Registries.DENSITY_FUNCTION)
.getOrThrow(ResourceKey.create(Registries.DENSITY_FUNCTION, Identifier.parse(functionId))).value();
return randomState.getSampler(function).sampleValue(densityContexts.get(), x, y, z);
}
public float[] densityChunk(DensityFunction function, int chunkX, int chunkZ) {
DensityVolume volume = new DensityVolume(CHUNK_WIDTH, heights.getHeight(), CHUNK_WIDTH, chunkX * CHUNK_WIDTH, heights.getMinY(), chunkZ * CHUNK_WIDTH);
return sampleVolume(function, volume);
}
public DensityFunction finalDensity() {
return generator.generatorSettings().value().noiseRouter().finalDensity();
}
public int baseHeight(int x, int z) {
return generator.getBaseHeight(x, z, Heightmap.Types.WORLD_SURFACE_WG, heights, randomState);
}
public int floorHeight(int x, int z) {
return generator.getBaseHeight(x, z, Heightmap.Types.OCEAN_FLOOR_WG, heights, randomState);
}
public ProtoChunk terrainChunk(int chunkX, int chunkZ) {
ProtoChunk chunk = new ProtoChunk(new ChunkPos(chunkX, chunkZ), UpgradeData.EMPTY, heights, containers, null);
NoiseGeneratorSettings settings = generator.generatorSettings().value();
NoiseSettings noise = settings.noiseSettings().clampToHeightAccessor(chunk.getHeightAccessorForGeneration());
try (NoiseChunk noiseChunk = (NoiseChunk) invoke(CREATE_NOISE_CHUNK, chunk, NoStructures.INSTANCE, Blender.empty(), randomState, noise)) {
invoke(DO_FILL, noiseChunk, chunk);
randomState.surfaceSystem().buildSurface(randomState, new BiomeManager(resolvers.get(), biomeZoomSeed),
new WorldGenerationContext(generator, chunk.getHeightAccessorForGeneration()), chunk, noiseChunk, settings.materialRule().value(), null);
}
return chunk;
}
public StructureStart structureStart(String structureId, int chunkX, int chunkZ) throws IOException {
Holder.Reference<Structure> structure = loaded.registries().lookupOrThrow(Registries.STRUCTURE)
.getOrThrow(ResourceKey.create(Registries.STRUCTURE, Identifier.parse(structureId)));
Path storageRoot = Files.createTempDirectory("iris-worldgen-export-structures-");
try (LevelStorageSource.LevelStorageAccess storage = LevelStorageSource.createDefault(storageRoot).createAccess("export")) {
StructureTemplateManager templates = new StructureTemplateManager(loaded.resources(), storage, DataFixers.getDataFixer(),
loaded.registries().lookupOrThrow(Registries.BLOCK));
return structure.value().generate(structure, Level.OVERWORLD, loaded.registries(), generator, generator.getBiomeSource(),
randomState.createClimateSampler(SamplerContext.builder().enableCaches().build()), randomState, templates, seed,
new ChunkPos(chunkX, chunkZ), 0, heights, structure.value().biomes()::contains);
} finally {
HeadlessNativeRegistries.delete(storageRoot);
}
}
public String biome(int x, int y, int z) {
Holder<Biome> biome = resolvers.get().getNoiseBiome(QuartPos.fromBlock(x), QuartPos.fromBlock(y), QuartPos.fromBlock(z));
return biome.unwrapKey().orElseThrow().identifier().toString();
}
public Set<String> possibleBiomes() {
Set<String> biomes = new TreeSet<>();
for (Holder<Biome> biome : generator.getBiomeSource().possibleBiomes()) {
biomes.add(biome.unwrapKey().orElseThrow().identifier().toString());
}
return biomes;
}
public SortedSet<String> densityFunctionIds() {
return densityFunctionIds;
}
public int minY() {
return heights.getMinY();
}
public int height() {
return heights.getHeight();
}
public int seaLevel() {
return generator.getSeaLevel();
}
public RegistryAccess registries() {
return loaded.registries();
}
@Override
public void close() throws IOException {
loaded.close();
}
private float[] sampleVolume(DensityFunction function, DensityVolume volume) {
try (ScopedDensityBuffer buffer = randomState.getSampler(function).bind(densityContexts.get()).sampleVolume(volume)) {
float[] values = new float[volume.size()];
for (int index = 0; index < values.length; index++) {
values[index] = buffer.get(index);
}
return values;
}
}
private Object invoke(Method method, Object... arguments) {
try {
return method.invoke(generator, arguments);
} catch (InvocationTargetException failure) {
if (failure.getCause() instanceof RuntimeException runtime) {
throw runtime;
}
if (failure.getCause() instanceof Error error) {
throw error;
}
throw new IllegalStateException("Terrain stage " + method.getName() + " failed", failure.getCause());
} catch (IllegalAccessException failure) {
throw new IllegalStateException("Terrain stage " + method.getName() + " is not accessible", failure);
}
}
private static Method generatorMethod(String name, Class<?>... parameters) {
try {
Method method = NoiseBasedChunkGenerator.class.getDeclaredMethod(name, parameters);
method.setAccessible(true);
return method;
} catch (NoSuchMethodException failure) {
throw new IllegalStateException("NoiseBasedChunkGenerator has no terrain stage " + name, failure);
}
}
private static LevelStem stem(RegistryAccess registries, WorldgenExportMode mode, Identifier dimension) {
return switch (mode) {
case PRESET -> registries.lookupOrThrow(Registries.WORLD_PRESET)
.getOrThrow(ResourceKey.create(Registries.WORLD_PRESET, dimension)).value().overworld()
.orElseThrow(() -> new IllegalStateException("World preset " + dimension + " has no overworld"));
case DIMENSION -> levelStem(registries, dimension);
case OVERWORLD -> levelStem(registries, LevelStem.OVERWORLD.identifier());
};
}
private static LevelStem levelStem(RegistryAccess registries, Identifier id) {
return registries.lookupOrThrow(Registries.LEVEL_STEM).getOrThrow(ResourceKey.create(Registries.LEVEL_STEM, id)).value();
}
private static SortedMap<String, byte[]> files(Path datapack) throws IOException {
if (!Files.isDirectory(datapack)) {
throw new IOException("Exported datapack " + datapack + " is not a directory");
}
SortedMap<String, byte[]> files = new TreeMap<>();
try (Stream<Path> paths = Files.walk(datapack)) {
List<Path> regular = paths.filter(Files::isRegularFile).toList();
for (Path file : regular) {
files.put(datapack.relativize(file).toString().replace('\\', '/'), Files.readAllBytes(file));
}
}
return files;
}
private static SortedSet<String> densityFunctionIds(Map<String, byte[]> files) {
SortedSet<String> ids = new TreeSet<>();
for (String path : files.keySet()) {
int marker = path.indexOf(DENSITY_FUNCTIONS);
if (path.startsWith("data/") && marker > 0 && path.endsWith(".json")) {
String namespace = path.substring("data/".length(), marker);
ids.add(namespace + ":" + path.substring(marker + DENSITY_FUNCTIONS.length(), path.length() - ".json".length()));
}
}
return ids;
}
private static final class NoStructures extends StructureManager {
private static final NoStructures INSTANCE = new NoStructures();
private NoStructures() {
super(null, null, null);
}
@Override
public List<StructureStart> startsForStructure(int chunkX, int chunkZ, Predicate<Structure> predicate) {
return List.of();
}
}
}
@@ -0,0 +1,394 @@
package art.arcane.iris.probe;
import io.papermc.paper.configuration.WorldConfiguration;
import net.minecraft.core.Holder;
import net.minecraft.core.QuartPos;
import net.minecraft.core.RegistryAccess;
import net.minecraft.core.registries.Registries;
import net.minecraft.resources.Identifier;
import net.minecraft.server.level.ServerLevel;
import net.minecraft.world.level.ChunkPos;
import net.minecraft.world.level.Level;
import net.minecraft.world.level.LevelHeightAccessor;
import net.minecraft.world.level.biome.Biome;
import net.minecraft.world.level.biome.BiomeManager;
import net.minecraft.world.level.biome.BiomeResolver;
import net.minecraft.world.level.chunk.ChunkAccess;
import net.minecraft.world.level.chunk.PalettedContainerFactory;
import net.minecraft.world.level.chunk.ProtoChunk;
import net.minecraft.world.level.chunk.UpgradeData;
import net.minecraft.world.level.chunk.status.ChunkStatus;
import net.minecraft.world.level.dimension.DimensionType;
import net.minecraft.world.level.dimension.LevelStem;
import net.minecraft.world.level.levelgen.Aquifer;
import net.minecraft.world.level.levelgen.Heightmap;
import net.minecraft.world.level.levelgen.NoiseBasedChunkGenerator;
import net.minecraft.world.level.levelgen.NoiseChunk;
import net.minecraft.world.level.levelgen.NoiseGeneratorSettings;
import net.minecraft.world.level.levelgen.NoiseSettings;
import net.minecraft.world.level.levelgen.RandomState;
import net.minecraft.world.level.levelgen.WorldGenerationContext;
import net.minecraft.world.level.levelgen.blending.Blender;
import net.minecraft.world.level.levelgen.densityfunction.DensityVolume;
import net.minecraft.world.level.levelgen.material.rule.MaterialRule;
import org.spigotmc.SpigotWorldConfig;
import java.lang.management.ManagementFactory;
import java.lang.management.ThreadMXBean;
import java.lang.reflect.Constructor;
import java.lang.reflect.Field;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.util.Arrays;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Map;
import java.util.Objects;
import java.util.Set;
import java.util.SplittableRandom;
import java.util.function.Supplier;
final class WorldgenStageBench {
private static final int SAMPLE_RADIUS_BLOCKS = 32_768;
private static final double NANOS_PER_MILLI = 1_000_000.0;
private static final double NANOS_PER_MICRO = 1_000.0;
private static final ThreadMXBean THREADS = ManagementFactory.getThreadMXBean();
private final String label;
private final NoiseBasedChunkGenerator generator;
private final NoiseGeneratorSettings settings;
private final RandomState randomState;
private final LevelHeightAccessor heights;
private final PalettedContainerFactory containers;
private final long seed;
private final Method fill;
private final Aquifer.FluidPicker fluidPicker;
private final BiomeResolver uncached;
private final Level surfaceLevel;
WorldgenStageBench(Options options) {
label = options.label();
if (!(options.stem().generator() instanceof NoiseBasedChunkGenerator noiseGenerator)) {
throw new IllegalStateException(label + " uses " + options.stem().generator().getClass().getSimpleName() + ", not a noise generator");
}
generator = noiseGenerator;
settings = noiseGenerator.generatorSettings().value();
seed = options.seed();
randomState = RandomState.create(options.registries().lookupOrThrow(Registries.NOISE), seed, settings);
DimensionType type = options.stem().type().value();
heights = LevelHeightAccessor.create(type.minY(), type.height());
containers = PalettedContainerFactory.create(options.registries());
fill = fillMethod();
fluidPicker = fluidPicker(noiseGenerator);
uncached = noiseGenerator.getBiomeSource().createUncachedResolver(randomState);
surfaceLevel = paperConfiguredLevel();
}
String label() {
return label;
}
NoiseGeneratorSettings settings() {
return settings;
}
void chunks(int originChunkX, int originChunkZ, int side, ChunkSamples samples) {
int ring = side + 2;
Map<Long, ProtoChunk> chunks = new HashMap<>(ring * ring * 2);
for (int dz = -1; dz <= side; dz++) {
for (int dx = -1; dx <= side; dx++) {
ChunkPos position = new ChunkPos(originChunkX + dx, originChunkZ + dz);
ProtoChunk chunk = new ProtoChunk(position, UpgradeData.EMPTY, heights, containers, null);
long started = System.nanoTime();
long startedCpu = THREADS.getCurrentThreadCpuTime();
generator.createBiomes(randomState, Blender.empty(), null, chunk).join();
samples.biomes.add(System.nanoTime() - started, THREADS.getCurrentThreadCpuTime() - startedCpu);
chunk.setPersistedStatus(ChunkStatus.BIOMES);
chunks.put(ChunkPos.pack(position.x(), position.z()), chunk);
}
}
RegionBiomes region = new RegionBiomes(chunks, uncached);
BiomeManager biomeManager = new BiomeManager(region, BiomeManager.obfuscateSeed(seed));
MaterialRule materialRule = settings.materialRule().value();
for (int dz = 0; dz < side; dz++) {
for (int dx = 0; dx < side; dx++) {
int chunkX = originChunkX + dx;
int chunkZ = originChunkZ + dz;
ProtoChunk chunk = chunks.get(ChunkPos.pack(chunkX, chunkZ));
Set<Holder<Biome>> possible = possibleBiomes(chunks, chunkX, chunkZ);
long started = System.nanoTime();
long startedCpu = THREADS.getCurrentThreadCpuTime();
try (NoiseChunk noiseChunk = new NoiseChunk(randomState, null, settings, fluidPicker, Blender.empty(), volume(chunk))) {
fill(noiseChunk, chunk);
long filled = System.nanoTime();
long filledCpu = THREADS.getCurrentThreadCpuTime();
randomState.surfaceSystem().buildSurface(randomState, biomeManager,
new WorldGenerationContext(generator, chunk.getHeightAccessorForGeneration(), surfaceLevel), chunk, noiseChunk, materialRule, possible);
samples.surface.add(System.nanoTime() - filled, THREADS.getCurrentThreadCpuTime() - filledCpu);
samples.noise.add(filled - started, filledCpu - startedCpu);
}
ChunkPos position = chunk.getPos();
long carverStarted = System.nanoTime();
long carverStartedCpu = THREADS.getCurrentThreadCpuTime();
uncached.getNoiseBiome(QuartPos.fromBlock(position.getMinBlockX()), 0, QuartPos.fromBlock(position.getMinBlockZ()));
samples.carverBiome.add(System.nanoTime() - carverStarted, THREADS.getCurrentThreadCpuTime() - carverStartedCpu);
samples.chunks++;
}
}
samples.biomeFallbacks += region.fallbacks();
}
void uncachedBiome(CallBudget budget, CallSamples samples) {
SplittableRandom random = new SplittableRandom(seed ^ 0x5DEECE66DL ^ samples.count);
int minQuart = QuartPos.fromBlock(heights.getMinY());
int quartHeight = QuartPos.fromBlock(heights.getHeight());
long deadline = System.nanoTime() + budget.limitNanos();
long startedCpu = THREADS.getCurrentThreadCpuTime();
int taken = 0;
while (taken < budget.calls() && (taken < budget.minimumCalls() || System.nanoTime() < deadline)) {
int quartX = QuartPos.fromBlock(random.nextInt(-SAMPLE_RADIUS_BLOCKS, SAMPLE_RADIUS_BLOCKS));
int quartY = minQuart + random.nextInt(quartHeight);
int quartZ = QuartPos.fromBlock(random.nextInt(-SAMPLE_RADIUS_BLOCKS, SAMPLE_RADIUS_BLOCKS));
long started = System.nanoTime();
uncached.getNoiseBiome(quartX, quartY, quartZ);
samples.wall.add(System.nanoTime() - started);
taken++;
}
samples.finish(taken, THREADS.getCurrentThreadCpuTime() - startedCpu);
}
void baseHeight(CallBudget budget, Heightmap.Types type, CallSamples samples) {
SplittableRandom random = new SplittableRandom(seed ^ type.ordinal() ^ 0x2545F4914F6CDD1DL ^ samples.count);
long deadline = System.nanoTime() + budget.limitNanos();
long startedCpu = THREADS.getCurrentThreadCpuTime();
int taken = 0;
while (taken < budget.calls() && (taken < budget.minimumCalls() || System.nanoTime() < deadline)) {
int x = random.nextInt(-SAMPLE_RADIUS_BLOCKS, SAMPLE_RADIUS_BLOCKS);
int z = random.nextInt(-SAMPLE_RADIUS_BLOCKS, SAMPLE_RADIUS_BLOCKS);
long started = System.nanoTime();
generator.getBaseHeight(x, z, type, heights, randomState);
samples.wall.add(System.nanoTime() - started);
taken++;
}
samples.finish(taken, THREADS.getCurrentThreadCpuTime() - startedCpu);
}
void baseColumn(CallBudget budget, CallSamples samples) {
SplittableRandom random = new SplittableRandom(seed ^ 0x9E3779B97F4A7C15L ^ samples.count);
long deadline = System.nanoTime() + budget.limitNanos();
long startedCpu = THREADS.getCurrentThreadCpuTime();
int taken = 0;
while (taken < budget.calls() && (taken < budget.minimumCalls() || System.nanoTime() < deadline)) {
int x = random.nextInt(-SAMPLE_RADIUS_BLOCKS, SAMPLE_RADIUS_BLOCKS);
int z = random.nextInt(-SAMPLE_RADIUS_BLOCKS, SAMPLE_RADIUS_BLOCKS);
long started = System.nanoTime();
generator.getBaseColumn(x, z, heights, randomState);
samples.wall.add(System.nanoTime() - started);
taken++;
}
samples.finish(taken, THREADS.getCurrentThreadCpuTime() - startedCpu);
}
private void fill(NoiseChunk noiseChunk, ChunkAccess chunk) {
try {
fill.invoke(generator, noiseChunk, chunk);
} catch (IllegalAccessException failure) {
throw new IllegalStateException("Cannot invoke NoiseBasedChunkGenerator.doFill for " + label, failure);
} catch (InvocationTargetException failure) {
if (failure.getCause() instanceof RuntimeException runtime) {
throw runtime;
}
throw new IllegalStateException("Noise fill failed for " + label, failure.getCause());
}
}
private DensityVolume volume(ChunkAccess chunk) {
NoiseSettings noise = settings.noiseSettings().clampToHeightAccessor(chunk.getHeightAccessorForGeneration());
ChunkPos position = chunk.getPos();
return new DensityVolume(16, noise.height(), 16, position.getMinBlockX(), noise.minY(), position.getMinBlockZ());
}
private static Set<Holder<Biome>> possibleBiomes(Map<Long, ProtoChunk> chunks, int chunkX, int chunkZ) {
Set<Holder<Biome>> possible = new HashSet<>();
for (int dz = -1; dz <= 1; dz++) {
for (int dx = -1; dx <= 1; dx++) {
chunks.get(ChunkPos.pack(chunkX + dx, chunkZ + dz)).collectBiomesInPalette(possible);
}
}
return possible;
}
private static Method fillMethod() {
try {
Method method = NoiseBasedChunkGenerator.class.getDeclaredMethod("doFill", NoiseChunk.class, ChunkAccess.class);
method.setAccessible(true);
return method;
} catch (NoSuchMethodException failure) {
throw new IllegalStateException("NoiseBasedChunkGenerator has no doFill(NoiseChunk, ChunkAccess)", failure);
}
}
@SuppressWarnings("unchecked")
private static Aquifer.FluidPicker fluidPicker(NoiseBasedChunkGenerator generator) {
try {
Field field = NoiseBasedChunkGenerator.class.getDeclaredField("globalFluidPicker");
field.setAccessible(true);
return ((Supplier<Aquifer.FluidPicker>) field.get(generator)).get();
} catch (ReflectiveOperationException failure) {
throw new IllegalStateException("NoiseBasedChunkGenerator has no globalFluidPicker supplier", failure);
}
}
private static Level paperConfiguredLevel() {
try {
Constructor<WorldConfiguration> constructor = WorldConfiguration.class.getDeclaredConstructor(SpigotWorldConfig.class, Identifier.class);
constructor.setAccessible(true);
WorldConfiguration configuration = constructor.newInstance(null, Identifier.withDefaultNamespace("overworld"));
configuration.environment = configuration.new Environment();
Class<?> unsafeType = Class.forName("sun.misc.Unsafe");
Field unsafeField = unsafeType.getDeclaredField("theUnsafe");
unsafeField.setAccessible(true);
Method allocate = unsafeType.getMethod("allocateInstance", Class.class);
Level level = (Level) allocate.invoke(unsafeField.get(null), ServerLevel.class);
Field paperConfig = Level.class.getDeclaredField("paperConfig");
paperConfig.setAccessible(true);
paperConfig.set(level, configuration);
return level;
} catch (ReflectiveOperationException failure) {
throw new IllegalStateException("Cannot build the Paper world configuration that surface rules read", failure);
}
}
record Options(String label, LevelStem stem, RegistryAccess registries, long seed) {
Options {
Objects.requireNonNull(label, "label");
Objects.requireNonNull(stem, "stem");
Objects.requireNonNull(registries, "registries");
}
}
record CallBudget(int calls, int minimumCalls, long limitNanos) {
}
record ChunkTimings(int chunks, StageTiming biomes, StageTiming noise, StageTiming surface, StageTiming carverBiome,
long biomeFallbacks) {
double totalMean() {
return biomes.wall().mean() + noise.wall().mean() + surface.wall().mean() + carverBiome.wall().mean();
}
double totalCpuMean() {
return biomes.cpu().mean() + noise.cpu().mean() + surface.cpu().mean() + carverBiome.cpu().mean();
}
}
record StageTiming(Summary wall, Summary cpu) {
}
record CallTiming(Summary wall, int calls, double cpuMean) {
}
record Summary(int count, double mean, double median, double p90, double min, double max) {
private static Summary of(long[] nanos, double unit) {
if (nanos.length == 0) {
return new Summary(0, Double.NaN, Double.NaN, Double.NaN, Double.NaN, Double.NaN);
}
long[] sorted = nanos.clone();
Arrays.sort(sorted);
double total = 0.0;
for (long value : sorted) {
total += value;
}
return new Summary(sorted.length, total / sorted.length / unit, percentile(sorted, 0.5) / unit,
percentile(sorted, 0.9) / unit, sorted[0] / unit, sorted[sorted.length - 1] / unit);
}
private static double percentile(long[] sorted, double fraction) {
int index = (int) Math.min(sorted.length - 1, Math.round(fraction * (sorted.length - 1)));
return sorted[index];
}
}
static final class ChunkSamples {
private final StageSamples biomes = new StageSamples();
private final StageSamples noise = new StageSamples();
private final StageSamples surface = new StageSamples();
private final StageSamples carverBiome = new StageSamples();
private int chunks;
private long biomeFallbacks;
ChunkTimings timings() {
return new ChunkTimings(chunks, biomes.timing(), noise.timing(), surface.timing(), carverBiome.timing(), biomeFallbacks);
}
}
static final class CallSamples {
private final Samples wall = new Samples();
private int count;
private long cpuNanos;
CallTiming timing() {
return new CallTiming(Summary.of(wall.values(), NANOS_PER_MICRO), count,
count == 0 ? Double.NaN : cpuNanos / (double) count / NANOS_PER_MICRO);
}
private void finish(int calls, long cpu) {
count += calls;
cpuNanos += cpu;
}
}
private static final class StageSamples {
private final Samples wall = new Samples();
private final Samples cpu = new Samples();
private void add(long wallNanos, long cpuNanos) {
wall.add(wallNanos);
cpu.add(cpuNanos);
}
private StageTiming timing() {
return new StageTiming(Summary.of(wall.values(), NANOS_PER_MILLI), Summary.of(cpu.values(), NANOS_PER_MILLI));
}
}
private static final class Samples {
private long[] values = new long[64];
private int size;
private void add(long value) {
if (size == values.length) {
values = Arrays.copyOf(values, size * 2);
}
values[size++] = value;
}
private long[] values() {
return Arrays.copyOf(values, size);
}
}
private static final class RegionBiomes implements BiomeResolver {
private final Map<Long, ProtoChunk> chunks;
private final BiomeResolver fallback;
private long fallbacks;
private RegionBiomes(Map<Long, ProtoChunk> chunks, BiomeResolver fallback) {
this.chunks = chunks;
this.fallback = fallback;
}
@Override
public Holder<Biome> getNoiseBiome(int quartX, int quartY, int quartZ) {
ProtoChunk chunk = chunks.get(ChunkPos.pack(QuartPos.toSection(quartX), QuartPos.toSection(quartZ)));
if (chunk == null) {
fallbacks++;
return fallback.getNoiseBiome(quartX, quartY, quartZ);
}
return chunk.getNoiseBiome(quartX, quartY, quartZ);
}
private long fallbacks() {
return fallbacks;
}
}
}
@@ -0,0 +1,61 @@
package art.arcane.iris.probe.fidelity;
import art.arcane.iris.probe.WorldgenExportSampler;
import java.io.IOException;
import java.util.Map;
import java.util.Objects;
public final class VanillaNoiseColumnSource implements ColumnSource {
private final WorldgenExportSampler sampler;
private final Map<String, String> loadKeyByBiomeId;
private final int minY;
private final int maxY;
private final int seaLevel;
public VanillaNoiseColumnSource(WorldgenExportSampler sampler, Map<String, String> loadKeyByBiomeId) {
this.sampler = Objects.requireNonNull(sampler, "sampler");
this.loadKeyByBiomeId = Map.copyOf(loadKeyByBiomeId);
this.minY = sampler.minY();
this.maxY = sampler.minY() + sampler.height();
this.seaLevel = sampler.seaLevel();
}
@Override
public String label() {
return "vanilla:26.3";
}
@Override
public int minY() {
return minY;
}
@Override
public int maxY() {
return maxY;
}
@Override
public FidelityColumn column(int x, int z) {
int floor = sampler.floorHeight(x, z);
int terrainY = floor - 1;
boolean wet = floor < seaLevel && sampler.baseHeight(x, z) > floor;
return new FidelityColumn(loadKey(x, Math.max(minY, terrainY), z), terrainY, wet);
}
@Override
public void close() throws IOException {
sampler.close();
}
private String loadKey(int x, int y, int z) {
String biomeId = sampler.biome(x, y, z);
String loadKey = loadKeyByBiomeId.get(biomeId);
if (loadKey == null) {
throw new IllegalStateException("Exported biome " + biomeId + " at " + x + "," + y + "," + z
+ " has no Iris load key in the export result");
}
return loadKey;
}
}
@@ -0,0 +1,204 @@
package art.arcane.iris.probe.fidelity;
import art.arcane.iris.probe.WorldgenExportSampler;
import net.minecraft.core.Holder;
import net.minecraft.core.registries.BuiltInRegistries;
import net.minecraft.world.level.block.Block;
import net.minecraft.world.level.chunk.LevelChunkSection;
import net.minecraft.world.level.chunk.ProtoChunk;
import net.minecraft.world.level.levelgen.densityfunction.DensityFunction;
import net.minecraft.world.level.levelgen.densityfunction.DensityFunctions;
import net.minecraft.world.level.levelgen.densityfunction.op.BinaryFunction;
import java.io.IOException;
import java.util.IdentityHashMap;
import java.util.Map;
import java.util.Objects;
public final class VanillaNoiseFidelitySource implements FidelitySource {
private static final int CHUNK_WIDTH = 16;
private static final byte SOLID = 0;
private static final byte OPEN = 1;
private static final byte FLUID = 2;
private static final byte LAVA = 3;
private final WorldgenExportSampler sampler;
private final Map<String, String> loadKeyByBiomeId;
private final String wallShell;
private final DensityFunction massDensity;
private final boolean walls;
private final int minY;
private final int maxY;
public VanillaNoiseFidelitySource(WorldgenExportSampler sampler, Map<String, String> loadKeyByBiomeId, String wallShell) {
this.sampler = Objects.requireNonNull(sampler, "sampler");
this.loadKeyByBiomeId = Map.copyOf(loadKeyByBiomeId);
this.wallShell = Objects.requireNonNull(wallShell, "wallShell");
this.massDensity = terrainMass(sampler.finalDensity());
this.walls = sampler.densityFunctionIds().contains(wallShell);
this.minY = sampler.minY();
this.maxY = sampler.minY() + sampler.height();
}
@Override
public String label() {
return "vanilla:26.3";
}
@Override
public int minY() {
return minY;
}
@Override
public int maxY() {
return maxY;
}
@Override
public FidelityChunk sample(int chunkX, int chunkZ) {
return new TerrainChunk(sampler.terrainChunk(chunkX, chunkZ), chunkX * CHUNK_WIDTH, chunkZ * CHUNK_WIDTH);
}
@Override
public void close() throws IOException {
sampler.close();
}
private String loadKey(int x, int y, int z) {
String biomeId = sampler.biome(x, y, z);
String loadKey = loadKeyByBiomeId.get(biomeId);
if (loadKey == null) {
throw new IllegalStateException("Exported biome " + biomeId + " at " + x + "," + y + "," + z
+ " has no Iris load key in the export result");
}
return loadKey;
}
static DensityFunction terrainMass(DensityFunction finalDensity) {
DensityFunction root = finalDensity;
while (root instanceof DensityFunctions.HolderHolder(Holder<DensityFunction> holder)) {
root = holder.value();
}
return root instanceof BinaryFunction(BinaryFunction.Type type, DensityFunction terrain, DensityFunction ignored) && type == BinaryFunction.Type.MIN
? terrain : root;
}
private static byte classify(Block block) {
String id = BuiltInRegistries.BLOCK.getKey(block).toString();
if (FidelityBlocks.open(id)) {
return OPEN;
}
if (FidelityBlocks.lava(id)) {
return LAVA;
}
return FidelityBlocks.fluid(id) ? FLUID : SOLID;
}
private record Column(int surfaceY, String biome, byte[] kinds) {
}
private final class TerrainChunk implements FidelityChunk {
private final ProtoChunk terrain;
private final int blockX;
private final int blockZ;
private final Column[] columns = new Column[CHUNK_WIDTH * CHUNK_WIDTH];
private final Map<Block, Byte> kinds = new IdentityHashMap<>();
private float[] masses;
private TerrainChunk(ProtoChunk terrain, int blockX, int blockZ) {
this.terrain = terrain;
this.blockX = blockX;
this.blockZ = blockZ;
}
@Override
public int surfaceY(int localX, int localZ) {
return column(localX, localZ).surfaceY();
}
@Override
public String surfaceBiome(int localX, int localZ) {
return column(localX, localZ).biome();
}
@Override
public boolean open(int localX, int y, int localZ) {
return kind(localX, y, localZ) == OPEN;
}
@Override
public boolean fluid(int localX, int y, int localZ) {
byte kind = kind(localX, y, localZ);
return kind == FLUID || kind == LAVA;
}
@Override
public boolean lava(int localX, int y, int localZ) {
return kind(localX, y, localZ) == LAVA;
}
@Override
public String block(int localX, int y, int localZ) {
return null;
}
@Override
public boolean terrainMass(int localX, int y, int localZ) {
if (y < minY || y >= maxY) {
throw new IllegalArgumentException("Y " + y + " is outside [" + minY + ", " + maxY + ")");
}
if (localX < 0 || localX >= CHUNK_WIDTH || localZ < 0 || localZ >= CHUNK_WIDTH) {
throw new IllegalArgumentException("Column " + localX + "," + localZ + " is outside the chunk");
}
if (masses == null) {
masses = sampler.densityChunk(massDensity, Math.floorDiv(blockX, CHUNK_WIDTH), Math.floorDiv(blockZ, CHUNK_WIDTH));
}
return masses[(localZ * CHUNK_WIDTH + localX) * (maxY - minY) + y - minY] > 0.0F;
}
@Override
public boolean wallsKnown() {
return walls;
}
@Override
public boolean wall(int localX, int y, int localZ) {
return walls && sampler.density(wallShell, blockX + localX, y, blockZ + localZ) > 0.0F;
}
private byte kind(int localX, int y, int localZ) {
if (y < minY || y >= maxY) {
throw new IllegalArgumentException("Y " + y + " is outside [" + minY + ", " + maxY + ")");
}
return column(localX, localZ).kinds()[y - minY];
}
private Column column(int localX, int localZ) {
if (localX < 0 || localX >= CHUNK_WIDTH || localZ < 0 || localZ >= CHUNK_WIDTH) {
throw new IllegalArgumentException("Column " + localX + "," + localZ + " is outside the chunk");
}
int index = localZ * CHUNK_WIDTH + localX;
Column column = columns[index];
if (column == null) {
column = sampleColumn(localX, localZ);
columns[index] = column;
}
return column;
}
private Column sampleColumn(int localX, int localZ) {
byte[] column = new byte[maxY - minY];
int surfaceY = minY - 1;
for (int y = maxY - 1; y >= minY; y--) {
LevelChunkSection section = terrain.getSection(terrain.getSectionIndex(y));
byte kind = kinds.computeIfAbsent(section.getBlockState(localX, y & (CHUNK_WIDTH - 1), localZ).getBlock(), VanillaNoiseFidelitySource::classify);
column[y - minY] = kind;
if (kind == SOLID && surfaceY < minY) {
surfaceY = y;
}
}
return new Column(surfaceY, loadKey(blockX + localX, Math.max(minY, surfaceY), blockZ + localZ), column);
}
}
}
@@ -0,0 +1,73 @@
package art.arcane.iris.probe;
import art.arcane.iris.platform.bukkit.BukkitPlatform;
import art.arcane.iris.spi.VanillaDataSource;
import net.minecraft.resources.Identifier;
import net.minecraft.server.packs.PackResources;
import net.minecraft.server.packs.PackType;
import net.minecraft.server.packs.repository.ServerPacksSource;
import net.minecraft.server.packs.resources.IoSupplier;
import org.bukkit.Bukkit;
import org.junit.BeforeClass;
import org.junit.Test;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import static org.junit.Assert.assertArrayEquals;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertNull;
import static org.junit.Assert.assertTrue;
public final class BukkitVanillaDataTest {
private static final String PLAINS = "data/minecraft/worldgen/biome/plains.json";
private static final String BIOME_TAGS = "data/minecraft/tags/worldgen/biome";
@BeforeClass
public static void bootstrap() {
HeadlessNativeBootstrap.initialize();
}
@Test
public void bukkitPlatformReadsTheServerPlainsBiome() throws IOException {
byte[] plains = new BukkitPlatform().vanillaData().read(PLAINS).orElseThrow();
assertTrue(new String(plains, StandardCharsets.UTF_8).contains("\"temperature\""));
try (PackResources vanilla = ServerPacksSource.createVanillaPackSource().fullResources()) {
IoSupplier<InputStream> resource = vanilla.getResource(PackType.SERVER_DATA,
Identifier.withDefaultNamespace("worldgen/biome/plains.json"));
try (InputStream input = resource.get()) {
assertArrayEquals(input.readAllBytes(), plains);
}
}
assertNull(Bukkit.getServer());
}
@Test
public void bukkitPlatformListsTheBiomeTagsTheGameLoads() {
List<String> expected = new ArrayList<>();
try (PackResources vanilla = ServerPacksSource.createVanillaPackSource().fullResources()) {
vanilla.listResources(PackType.SERVER_DATA, "minecraft", "tags/worldgen/biome",
(id, resource) -> expected.add("data/" + id.getNamespace() + "/" + id.getPath()));
}
Collections.sort(expected);
List<String> listed = new BukkitPlatform().vanillaData().list(BIOME_TAGS);
assertFalse(listed.isEmpty());
assertTrue(listed.contains(BIOME_TAGS + "/is_overworld.json"));
assertEquals(expected, listed);
}
@Test
public void headlessPlatformReadsTheSameVanillaData() {
HeadlessNativePlatform platform = new HeadlessNativePlatform(new File("unused-native-platform"),
HeadlessNativeTestRegistries::get);
VanillaDataSource bukkit = new BukkitPlatform().vanillaData();
assertArrayEquals(bukkit.read(PLAINS).orElseThrow(), platform.vanillaData().read(PLAINS).orElseThrow());
assertEquals(bukkit.list(BIOME_TAGS), platform.vanillaData().list(BIOME_TAGS));
}
}
@@ -0,0 +1,146 @@
package art.arcane.iris.probe;
import net.minecraft.core.RegistryAccess;
import net.minecraft.core.registries.Registries;
import net.minecraft.resources.Identifier;
import net.minecraft.resources.ResourceKey;
import net.minecraft.world.level.levelgen.NoiseBasedChunkGenerator;
import net.minecraft.world.level.levelgen.NoiseGeneratorSettings;
import net.minecraft.world.level.levelgen.RandomState;
import net.minecraft.world.level.levelgen.densityfunction.CachingDensitySampler;
import net.minecraft.world.level.levelgen.densityfunction.DensityFunction;
import net.minecraft.world.level.levelgen.densityfunction.DensitySampler;
import net.minecraft.world.level.levelgen.material.rule.ConditionRule;
import net.minecraft.world.level.levelgen.material.rule.MaterialRule;
import net.minecraft.world.level.levelgen.material.rule.OreVeinRule;
import net.minecraft.world.level.levelgen.material.rule.SequenceRule;
import java.lang.reflect.Field;
import java.lang.reflect.Modifier;
import java.util.ArrayDeque;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.Deque;
import java.util.IdentityHashMap;
import java.util.List;
import java.util.Set;
import java.util.TreeSet;
final class CompiledCaches {
private static final String MINECRAFT_PACKAGE = "net.minecraft.";
private static final String DENSITY_PACKAGE = "net.minecraft.world.level.levelgen.densityfunction.";
private static final String SPLINE_PACKAGE = "net.minecraft.util.";
private static final String SPLINE_NAME = "Spline";
private final RegistryAccess registries;
private final NoiseGeneratorSettings settings;
private final RandomState state;
private CompiledCaches(RegistryAccess registries, NoiseGeneratorSettings settings, long seed) {
this.registries = registries;
this.settings = settings;
this.state = RandomState.create(registries.lookupOrThrow(Registries.NOISE), seed, settings);
}
static CompiledCaches open(RegistryAccess registries, long seed) {
if (!(DatapackCostProbe.exportedOverworld(registries).generator() instanceof NoiseBasedChunkGenerator generator)) {
throw new IllegalStateException("Exported overworld has no noise generator");
}
return new CompiledCaches(registries, generator.generatorSettings().value(), seed);
}
DensityFunction finalDensity() {
return settings.noiseRouter().finalDensity();
}
DensityFunction function(String id) {
return registries.lookupOrThrow(Registries.DENSITY_FUNCTION)
.getOrThrow(ResourceKey.create(Registries.DENSITY_FUNCTION, Identifier.parse(id))).value();
}
List<OreVeinRule> oreVeins() {
List<OreVeinRule> veins = new ArrayList<>();
Deque<MaterialRule> pending = new ArrayDeque<>();
pending.push(settings.materialRule().value());
while (!pending.isEmpty()) {
MaterialRule rule = pending.pop();
switch (rule) {
case OreVeinRule vein -> veins.add(vein);
case SequenceRule sequence -> sequence.sequence().reversed().forEach(pending::push);
case ConditionRule condition -> pending.push(condition.thenRun());
default -> {
}
}
}
return veins;
}
int rootId(DensityFunction function) {
if (!(state.getSampler(function) instanceof CachingDensitySampler cache)) {
throw new IllegalStateException("Density function " + function + " does not compile to a cache");
}
return cache.id();
}
Set<Integer> ids(DensityFunction function) {
Set<Integer> ids = new TreeSet<>();
Set<Object> visited = Collections.newSetFromMap(new IdentityHashMap<>());
Deque<Object> pending = new ArrayDeque<>();
pending.push(state.getSampler(function));
while (!pending.isEmpty()) {
Object node = pending.pop();
if (node == null || !visited.add(node)) {
continue;
}
if (node instanceof CachingDensitySampler cache) {
ids.add(cache.id());
}
for (Object child : children(node)) {
pending.push(child);
}
}
return ids;
}
private static List<Object> children(Object node) {
List<Object> children = new ArrayList<>();
if (node instanceof Object[] array) {
Collections.addAll(children, array);
return children;
}
if (node instanceof Collection<?> collection) {
children.addAll(collection);
return children;
}
if (!traversed(node)) {
return children;
}
for (Class<?> type = node.getClass(); type != null && type.getName().startsWith(MINECRAFT_PACKAGE); type = type.getSuperclass()) {
for (Field field : type.getDeclaredFields()) {
if (Modifier.isStatic(field.getModifiers()) || field.getType().isPrimitive()) {
continue;
}
children.add(read(field, node));
}
}
return children;
}
private static boolean traversed(Object node) {
if (node instanceof Enum<?>) {
return false;
}
String name = node.getClass().getName();
return node instanceof DensitySampler || name.startsWith(DENSITY_PACKAGE) || name.startsWith(SPLINE_PACKAGE) && name.contains(SPLINE_NAME);
}
private static Object read(Field field, Object owner) {
try {
field.setAccessible(true);
return field.get(owner);
} catch (IllegalAccessException | RuntimeException failure) {
throw new IllegalStateException("Cannot read " + field + " of " + owner.getClass().getName(), failure);
}
}
}
@@ -0,0 +1,46 @@
package art.arcane.iris.probe;
import art.arcane.iris.probe.fidelity.FidelityReport;
import com.google.gson.JsonObject;
import com.google.gson.JsonParser;
import org.junit.ClassRule;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.TemporaryFolder;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.List;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
public final class DatapackFidelityProbeBiomeModeTest {
@ClassRule
public static final PaperRegistryIsolation registryIsolation = new PaperRegistryIsolation();
@Rule
public TemporaryFolder temporary = new TemporaryFolder();
@Test(timeout = 300_000L)
public void biomeModeJudgesTheSeedMeanWithoutTheChunkScan() throws Exception {
Path pack = DatapackFidelityProbeTest.pack(temporary);
Path output = temporary.getRoot().toPath().resolve("reports").resolve("biomes.json");
FidelityReport report = DatapackFidelityProbe.execute(
new String[]{pack.toString(), "main", "1337", "0", "1", output.toString(), "256", "32", "3", "biomes"}, DatapackFidelityProbeTest.exporter());
JsonObject json = JsonParser.parseString(Files.readString(output, StandardCharsets.UTF_8)).getAsJsonObject();
assertTrue(report.toTable(), report.passed());
assertTrue(json.get("passed").getAsBoolean());
assertEquals(1.0D, DatapackFidelityProbeTest.metric(report, "biome.share.plains").export(), 0.0D);
assertEquals(1, report.sampling().size());
assertEquals(List.of(1337L, 1338L, 1339L), report.sampling().getFirst().seeds());
assertEquals(3, json.getAsJsonArray("seeds").size());
assertEquals(3, json.getAsJsonObject("seedsPassing").getAsJsonObject("biome.share.plains").get("passing").getAsInt());
assertTrue(report.observations().stream().anyMatch(observation -> observation.name().equals("biome.type.land")
&& observation.iris() == 1.0D && observation.export() == 1.0D));
assertTrue(RealPackProbeSupport.drainReported().isEmpty());
}
}
@@ -0,0 +1,116 @@
package art.arcane.iris.probe;
import art.arcane.iris.pack.datapack.export.WorldgenDatapackExporter;
import art.arcane.iris.pack.datapack.export.compile.DimensionStage;
import art.arcane.iris.pack.datapack.export.compile.ImageMapStage;
import art.arcane.iris.pack.datapack.export.compile.PackCompiler;
import art.arcane.iris.pack.datapack.export.emit.WorldgenJsonWriter;
import art.arcane.iris.probe.fidelity.FidelityMetric;
import art.arcane.iris.probe.fidelity.FidelityReport;
import com.google.gson.JsonObject;
import com.google.gson.JsonParser;
import org.junit.ClassRule;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.TemporaryFolder;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.List;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertThrows;
import static org.junit.Assert.assertTrue;
public final class DatapackFidelityProbeTest {
@ClassRule
public static final PaperRegistryIsolation registryIsolation = new PaperRegistryIsolation();
@Rule
public TemporaryFolder temporary = new TemporaryFolder();
@Test(timeout = 300_000L)
public void probeExportsTheSyntheticPackAndComparesBothGenerators() throws Exception {
Path pack = pack(temporary);
Path output = temporary.getRoot().toPath().resolve("reports").resolve("fidelity.json");
FidelityReport report = DatapackFidelityProbe.execute(
new String[]{pack.toString(), "main", "1337", "1", "8", output.toString(), "256", "32", "2", "full"}, exporter());
JsonObject json = JsonParser.parseString(Files.readString(output, StandardCharsets.UTF_8)).getAsJsonObject();
assertTrue(report.toTable(), report.passed());
assertTrue(json.get("passed").getAsBoolean());
assertEquals(report.metrics().size(), json.getAsJsonArray("metrics").size());
FidelityMetric biome = metric(report, "biome.share.plains");
assertEquals(1.0D, biome.iris(), 0.0D);
assertEquals(1.0D, biome.export(), 0.0D);
assertTrue(biome.pass());
assertEquals(0.0D, metric(report, "biome.totalVariation").delta(), 0.0D);
assertEquals(1.0D, metric(report, "height.landFraction").export(), 0.0D);
JsonObject columns = json.getAsJsonArray("sampling").get(0).getAsJsonObject();
assertEquals("columns", columns.get("scope").getAsString());
assertEquals(256, columns.get("windowBlocks").getAsInt());
assertEquals(32, columns.get("strideBlocks").getAsInt());
assertEquals(2, columns.getAsJsonArray("seeds").size());
assertEquals(128L, columns.get("columns").getAsLong());
JsonObject chunks = json.getAsJsonArray("sampling").get(1).getAsJsonObject();
assertEquals("chunks", chunks.get("scope").getAsString());
assertEquals(48, chunks.get("windowBlocks").getAsInt());
assertTrue(RealPackProbeSupport.drainReported().isEmpty());
}
@Test
public void malformedArgumentsAreRejectedBeforeThePackOpens() throws Exception {
Path pack = pack(temporary);
String output = temporary.getRoot().toPath().resolve("unused.json").toString();
IllegalArgumentException missing = assertThrows(IllegalArgumentException.class,
() -> DatapackFidelityProbe.execute(new String[]{pack.toString(), "main"}, exporter()));
IllegalArgumentException stride = assertThrows(IllegalArgumentException.class,
() -> DatapackFidelityProbe.execute(new String[]{pack.toString(), "main", "1337", "1", "17", output, "256", "32", "1", "full"}, exporter()));
IllegalArgumentException survey = assertThrows(IllegalArgumentException.class,
() -> DatapackFidelityProbe.execute(new String[]{pack.toString(), "main", "1337", "1", "8", output, "250", "32", "1", "full"}, exporter()));
IllegalArgumentException seeds = assertThrows(IllegalArgumentException.class,
() -> DatapackFidelityProbe.execute(new String[]{pack.toString(), "main", "1337", "1", "8", output, "256", "32", "0", "full"}, exporter()));
IllegalArgumentException mode = assertThrows(IllegalArgumentException.class,
() -> DatapackFidelityProbe.execute(new String[]{pack.toString(), "main", "1337", "1", "8", output, "256", "32", "1", "caves"}, exporter()));
assertTrue(missing.getMessage(), missing.getMessage().startsWith("Usage:"));
assertTrue(stride.getMessage(), stride.getMessage().contains("stride"));
assertTrue(survey.getMessage(), survey.getMessage().contains("multiple"));
assertTrue(seeds.getMessage(), seeds.getMessage().contains("surveySeeds"));
assertTrue(mode.getMessage(), mode.getMessage().contains("caves"));
assertFalse(Files.exists(Path.of(output)));
}
static WorldgenDatapackExporter exporter() {
return new WorldgenDatapackExporter(WorldgenJsonWriter::forTarget, new PackCompiler(List.of(new DimensionStage(), new ImageMapStage())));
}
static FidelityMetric metric(FidelityReport report, String name) {
for (FidelityMetric metric : report.metrics()) {
if (metric.name().equals(name)) {
return metric;
}
}
throw new AssertionError("Report has no metric " + name + ": " + report.toJson());
}
static Path pack(TemporaryFolder temporary) throws Exception {
Path pack = temporary.newFolder().toPath();
for (String directory : List.of("dimensions", "regions", "biomes", "generators")) {
Files.createDirectories(pack.resolve(directory));
}
Files.writeString(pack.resolve("dimensions/main.json"), """
{"name":"Fidelity","focus":"plains","regions":["owned"],"landChance":1,"carvingEnabled":false}
""");
Files.writeString(pack.resolve("regions/owned.json"), "{\"landBiomes\":[\"plains\"]}");
Files.writeString(pack.resolve("generators/flat.json"), "{\"composite\":[]}");
Files.writeString(pack.resolve("biomes/plains.json"), """
{"derivative":"minecraft:plains","generators":[{"generator":"flat","min":0,"max":0}]}
""");
return pack;
}
}
@@ -0,0 +1,110 @@
package art.arcane.iris.probe;
import net.minecraft.core.registries.Registries;
import net.minecraft.resources.Identifier;
import net.minecraft.resources.ResourceKey;
import net.minecraft.world.level.levelgen.densityfunction.DensityFunction;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.ClassRule;
import org.junit.Test;
import java.io.IOException;
import java.util.LinkedHashMap;
import java.util.Map;
import static org.junit.Assert.assertEquals;
public final class DensityCostAnalysisTest {
@ClassRule
public static final PaperRegistryIsolation registryIsolation = new PaperRegistryIsolation();
private static final double EPSILON = 1.0E-9;
private static final String LEAF = "{\"type\":\"noise\",\"noise\":\"minecraft:temperature\",\"xz_scale\":1.0,\"y_scale\":0.0}";
private static HeadlessNativeRegistries registries;
@BeforeClass
public static void loadGraphs() throws IOException {
Map<String, String> definitions = new LinkedHashMap<>();
definitions.put(path("leaf"), LEAF);
definitions.put(path("pair"), "{\"type\":\"add\",\"left\":\"t:leaf\",\"right\":\"t:leaf\"}");
definitions.put(path("quad"), "{\"type\":\"mul\",\"left\":\"t:pair\",\"right\":\"t:pair\"}");
definitions.put(path("cached"), "{\"type\":\"cache\",\"input\":\"t:pair\"}");
definitions.put(path("twice"), "{\"type\":\"add\",\"left\":\"t:cached\",\"right\":\"t:cached\"}");
definitions.put(path("select"), "{\"type\":\"interval_select\",\"input\":\"t:leaf\",\"thresholds\":[0.0],\"functions\":[\"t:quad\",\"t:pair\"]}");
definitions.put(path("spline"), "{\"type\":\"spline\",\"spline\":{\"coordinate\":\"t:leaf\",\"points\":["
+ "{\"location\":-1.0,\"value\":" + nested("t:quad") + ",\"derivative\":0.0},"
+ "{\"location\":0.0,\"value\":0.0,\"derivative\":0.0},"
+ "{\"location\":1.0,\"value\":" + nested("t:quad") + ",\"derivative\":0.0}]}}");
definitions.put(path("grid"), "{\"type\":\"interpolated\",\"input\":\"t:pair\",\"cell_size_xz\":4,\"cell_size_y\":8}");
registries = HeadlessNativeRegistries.load(definitions);
}
@AfterClass
public static void closeRegistries() throws IOException {
if (registries != null) {
registries.close();
}
}
@Test
public void referencesExpandWhileTheDagSharesThem() {
DensityCostAnalysis.RootCost quad = analysis().analyze("quad", function("quad"));
assertEquals(3L, quad.dagNodes());
assertEquals(7.0, quad.expandedNodes(), EPSILON);
assertEquals(7.0, quad.cachedNodes(), EPSILON);
assertEquals(2, quad.namedSubtrees());
assertEquals("t:leaf", quad.exclusive().getFirst().name());
assertEquals(4.0, quad.exclusive().getFirst().multiplicity(), EPSILON);
assertEquals(4.0, quad.exclusive().getFirst().contribution(), EPSILON);
assertEquals("t:pair", quad.inclusive().getFirst().name());
assertEquals(6.0, quad.inclusive().getFirst().inclusive(), EPSILON);
assertEquals(1.0 / 3.0, quad.dagShare().get("leaf"), EPSILON);
assertEquals(1.0 / 3.0, quad.dagShare().get("quad"), EPSILON);
}
@Test
public void cachesEvaluateTheirBodyOnceWhenEnabled() {
DensityCostAnalysis.RootCost twice = analysis().analyze("twice", function("twice"));
assertEquals(4L, twice.dagNodes());
assertEquals(9.0, twice.expandedNodes(), EPSILON);
assertEquals(6.0, twice.cachedNodes(), EPSILON);
}
@Test
public void uncachedPointEvaluationTakesOneSelectedBranch() {
DensityCostAnalysis analysis = analysis();
DensityCostAnalysis.RootCost select = analysis.analyze("select", function("select"));
assertEquals(12.0, select.expandedNodes(), EPSILON);
assertEquals(9.0, select.uncachedPointWorst(), EPSILON);
DensityCostAnalysis.RootCost spline = analysis.analyze("spline", function("spline"));
assertEquals(19.0, spline.expandedNodes(), EPSILON);
assertEquals(11.0, spline.uncachedPointWorst(), EPSILON);
}
@Test
public void offLatticeGridPointsSampleEightCorners() {
DensityCostAnalysis.RootCost grid = analysis().analyze("grid", function("grid"));
assertEquals(4.0, grid.expandedNodes(), EPSILON);
assertEquals(25.0, grid.uncachedPointWorst(), EPSILON);
}
private static DensityCostAnalysis analysis() {
return new DensityCostAnalysis(id -> DensityJsonBreakdown.category("overworld", id), 10);
}
private static DensityFunction function(String name) {
return registries.registries().lookupOrThrow(Registries.DENSITY_FUNCTION)
.getOrThrow(ResourceKey.create(Registries.DENSITY_FUNCTION, Identifier.parse("t:" + name))).value();
}
private static String nested(String coordinate) {
return "{\"coordinate\":\"" + coordinate + "\",\"points\":[{\"location\":0.0,\"value\":1.0,\"derivative\":0.0},"
+ "{\"location\":1.0,\"value\":2.0,\"derivative\":0.0}]}";
}
private static String path(String name) {
return "data/t/worldgen/density_function/" + name + ".json";
}
}
@@ -0,0 +1,88 @@
package art.arcane.iris.probe;
import org.junit.Test;
import java.nio.charset.StandardCharsets;
import java.util.SortedMap;
import java.util.TreeMap;
import static org.junit.Assert.assertEquals;
public final class DensityJsonBreakdownTest {
private static final String TEMPERATURE = "{\"type\":\"add\",\"left\":\"x:biome/selector/0\",\"right\":1.0}";
private static final String SELECTOR = "{\"type\":\"mul\",\"left\":\"x:cell/cellular/a/value\",\"right\":2.0}";
private static final String CELL_A = "{\"type\":\"noise\",\"noise\":\"x:hash_a\",\"xz_scale\":0.5,\"y_scale\":0.0}";
private static final String CELL_B = "{\"type\":\"noise\",\"noise\":\"x:hash_b\",\"xz_scale\":0.25,\"y_scale\":0.0}";
private static final String FINAL_DENSITY = "{\"type\":\"min\",\"left\":\"x:cell/cellular/b/value\",\"right\":0.0}";
private static final String NOISE = "{\"firstOctave\":-1,\"amplitudes\":[1.0]}";
@Test
public void categoriesFollowProducingCompilers() {
assertEquals("router", DensityJsonBreakdown.category("overworld", "x:overworld/final_density"));
assertEquals("cellular", DensityJsonBreakdown.category("overworld", "x:cell/cracks/abc/distance"));
assertEquals("biome selector", DensityJsonBreakdown.category("overworld", "x:biome/selector/12"));
assertEquals("biome decision", DensityJsonBreakdown.category("overworld", "x:biome/base_code"));
assertEquals("terrain height", DensityJsonBreakdown.category("overworld", "x:height"));
assertEquals("terrain bands", DensityJsonBreakdown.category("overworld", "x:terrain/plains/blend/3"));
assertEquals("noise style", DensityJsonBreakdown.category("overworld", "x:style/shared/ab12"));
assertEquals("cellular offset copies", DensityJsonBreakdown.category("overworld", "x:cell/cellular/abc/vx_0_0/offset_x4_z0"));
assertEquals("biome selector offset copies", DensityJsonBreakdown.category("overworld", "x:biome/selector/4/offset_x0_z4"));
}
@Test
public void reportsBytesShapesAndCellularOwners() {
SortedMap<String, byte[]> files = new TreeMap<>();
put(files, "density_function/overworld/temperature", TEMPERATURE);
put(files, "density_function/biome/selector/0", SELECTOR);
put(files, "density_function/cell/cellular/a/value", CELL_A);
put(files, "density_function/cell/cellular/b/value", CELL_B);
put(files, "density_function/overworld/final_density", FINAL_DENSITY);
put(files, "noise/hash_a", NOISE);
put(files, "noise/hash_b", NOISE);
DensityJsonBreakdown.Report report = DensityJsonBreakdown.analyze(files, "overworld").report();
assertEquals(5, report.files());
assertEquals(bytes(TEMPERATURE, SELECTOR, CELL_A, CELL_B, FINAL_DENSITY), report.bytes());
DensityJsonBreakdown.CategoryBytes cellular = category(report, "cellular");
assertEquals(2, cellular.files());
assertEquals(1, cellular.distinctShapes());
assertEquals(bytes(CELL_B), cellular.structurallyDuplicateBytes());
assertEquals(2, category(report, "router").files());
assertEquals(2, report.cellularInstances());
assertEquals(1, report.cellularShapes().size());
assertEquals("biome selector+router", report.cellularShapes().getFirst().owners());
assertEquals(bytes(CELL_A), owner(report, "cellular via biome selector"));
assertEquals(bytes(CELL_B), owner(report, "cellular via router"));
}
private static DensityJsonBreakdown.CategoryBytes category(DensityJsonBreakdown.Report report, String name) {
for (DensityJsonBreakdown.CategoryBytes category : report.categories()) {
if (category.category().equals(name)) {
return category;
}
}
throw new AssertionError("No category " + name + " in " + report.categories());
}
private static long owner(DensityJsonBreakdown.Report report, String name) {
for (DensityJsonBreakdown.Bytes owner : report.libraryOwners()) {
if (owner.name().equals(name)) {
return owner.bytes();
}
}
throw new AssertionError("No owner " + name + " in " + report.libraryOwners());
}
private static long bytes(String... bodies) {
long total = 0;
for (String body : bodies) {
total += body.getBytes(StandardCharsets.UTF_8).length;
}
return total;
}
private static void put(SortedMap<String, byte[]> files, String path, String body) {
files.put("data/x/worldgen/" + path + ".json", body.getBytes(StandardCharsets.UTF_8));
}
}
@@ -1,37 +1,19 @@
package art.arcane.iris.probe; package art.arcane.iris.probe;
import io.papermc.paper.registry.PaperRegistryAccess;
import org.junit.rules.TestRule; import org.junit.rules.TestRule;
import org.junit.runner.Description; import org.junit.runner.Description;
import org.junit.runners.model.Statement; import org.junit.runners.model.Statement;
import java.lang.reflect.Field;
import java.util.HashMap;
import java.util.Map;
public final class PaperRegistryIsolation implements TestRule { public final class PaperRegistryIsolation implements TestRule {
@Override @Override
public Statement apply(Statement base, Description description) { public Statement apply(Statement base, Description description) {
return new Statement() { return new Statement() {
@Override @Override
public void evaluate() throws Throwable { public void evaluate() throws Throwable {
Map<Object, Object> registries = registries(); try (PaperRegistryScope scope = PaperRegistryScope.open()) {
Map<Object, Object> previous = new HashMap<>(registries);
registries.clear();
try {
base.evaluate(); base.evaluate();
} finally {
registries.clear();
registries.putAll(previous);
} }
} }
}; };
} }
@SuppressWarnings("unchecked")
private static Map<Object, Object> registries() throws ReflectiveOperationException {
Field field = PaperRegistryAccess.class.getDeclaredField("registries");
field.setAccessible(true);
return (Map<Object, Object>) field.get(PaperRegistryAccess.instance());
}
} }
@@ -0,0 +1,126 @@
package art.arcane.iris.probe;
import art.arcane.iris.pack.datapack.DataVersion;
import art.arcane.iris.pack.datapack.export.emit.DensityFunctionEmitter;
import art.arcane.iris.pack.datapack.export.emit.FunctionTable;
import art.arcane.iris.pack.datapack.export.ir.Axis;
import art.arcane.iris.pack.datapack.export.ir.Field;
import art.arcane.iris.pack.datapack.export.ir.NoiseDef;
import art.arcane.iris.pack.datapack.export.ir.ValueRange;
import art.arcane.iris.pack.datapack.export.lower.ExportCapabilities;
import art.arcane.iris.pack.datapack.export.lower.VanillaBounds;
import com.google.gson.JsonArray;
import com.google.gson.JsonElement;
import com.google.gson.JsonObject;
import net.minecraft.core.registries.Registries;
import net.minecraft.resources.Identifier;
import net.minecraft.resources.ResourceKey;
import net.minecraft.util.Interval;
import org.junit.ClassRule;
import org.junit.Test;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
public final class VanillaBoundsParity263Test {
@ClassRule
public static final PaperRegistryIsolation registryIsolation = new PaperRegistryIsolation();
private static final String NAMESPACE = "iris_export";
private static final List<NoiseDef> NOISES = List.of(new NoiseDef("parity/base", -3, List.of(1.0, 0.5, 0.25)),
new NoiseDef("parity/sparse", 0, List.of(0.0, 1.5, 0.0, 0.7)));
private static final Field BASE = new Field.Noise(NAMESPACE + ":parity/base", 0.25, 0.0, null, null, null);
private static final Field SPARSE = new Field.Noise(NAMESPACE + ":parity/sparse", 0.5, 0.25, null, null, null);
private static final Field UNIT = new Field.Clamp(BASE, 0.0, 1.0);
private static final Field BAND = new Field.Clamp(SPARSE, 0.3, 0.9);
@Test
public void boundsMatchMinecraftForEveryEmittedNode() throws Exception {
List<Field> fields = fields();
VanillaBounds bounds = VanillaBounds.of(NAMESPACE, NOISES);
DensityFunctionEmitter emitter = DensityFunctionEmitter.forTarget(ExportCapabilities.forTarget(DataVersion.V26_3), NAMESPACE, bounds);
FunctionTable table = new FunctionTable(NAMESPACE);
List<String> references = new ArrayList<>(fields.size());
for (int index = 0; index < fields.size(); index++) {
JsonElement emitted = emitter.emit(fields.get(index), table);
boolean reference = emitted.isJsonPrimitive() && emitted.getAsJsonPrimitive().isString();
references.add(reference ? emitted.getAsString() : table.register("parity/field_" + index, emitted));
}
Map<String, String> definitions = new LinkedHashMap<>();
for (Map.Entry<String, JsonElement> function : table.files().entrySet()) {
definitions.put("data/" + NAMESPACE + "/worldgen/density_function/" + function.getKey() + ".json", function.getValue().toString());
}
for (NoiseDef noise : NOISES) {
definitions.put("data/" + NAMESPACE + "/worldgen/noise/" + noise.id() + ".json", noiseJson(noise).toString());
}
try (HeadlessNativeRegistries registries = HeadlessNativeRegistries.load(definitions)) {
for (int index = 0; index < fields.size(); index++) {
Interval vanilla = registries.registries().lookupOrThrow(Registries.DENSITY_FUNCTION)
.getOrThrow(ResourceKey.create(Registries.DENSITY_FUNCTION, Identifier.parse(references.get(index)))).value().range();
Optional<ValueRange> modelled = bounds.range(fields.get(index));
assertTrue("field " + index + " " + fields.get(index) + " has no modelled range, Minecraft " + vanilla, modelled.isPresent());
assertEquals("field " + index + " " + fields.get(index), new ValueRange(vanilla.min(), vanilla.max()), modelled.get());
}
}
}
private static List<Field> fields() {
Field scaled = new Field.Binary(Field.BinaryOp.MUL, new Field.Const(0.3), BAND);
Field curve = new Field.Curve(new Field.Binary(Field.BinaryOp.MUL, BASE, new Field.Const(1.7)), List.of(
new Field.Knot(-1.0, new Field.Const(0.2), 0.75),
new Field.Knot(0.1, new Field.Curve(SPARSE, List.of(new Field.Knot(-0.5, new Field.Const(-1.0), 0.0),
new Field.Knot(0.5, new Field.Const(1.3), 2.0))), 0.0),
new Field.Knot(0.8, new Field.Const(0.6), -1.5)));
List<Field> fields = new ArrayList<>();
fields.add(BASE);
fields.add(SPARSE);
fields.add(new Field.Const(0.1));
fields.add(new Field.Coord(Axis.Y, Field.Tiling.CLAMP, -64.0, 320.0, 1.5, -0.7));
fields.add(new Field.Clamp(BASE, -0.3, 0.8));
for (Field.BinaryOp op : Field.BinaryOp.values()) {
fields.add(new Field.Binary(op, BASE, new Field.Binary(Field.BinaryOp.ADD, BAND, new Field.Const(0.6))));
}
fields.add(new Field.Binary(Field.BinaryOp.POW, UNIT, new Field.Const(2.5)));
fields.add(new Field.Binary(Field.BinaryOp.MAX, scaled, new Field.Const(0.0)));
fields.add(new Field.Binary(Field.BinaryOp.MIN, new Field.Const(2.0), scaled));
fields.add(new Field.Binary(Field.BinaryOp.MIN, new Field.Const(0.2), scaled));
for (Field.UnaryOp op : Field.UnaryOp.values()) {
fields.add(new Field.Unary(op, new Field.Binary(Field.BinaryOp.SUB, SPARSE, new Field.Const(0.4))));
}
fields.add(new Field.Lerp(UNIT, BASE, new Field.Binary(Field.BinaryOp.MUL, SPARSE, new Field.Const(3.0))));
for (Field.RoundMode mode : Field.RoundMode.values()) {
fields.add(new Field.Round(mode, new Field.Binary(Field.BinaryOp.MUL, BASE, new Field.Const(10.3)), new Field.Const(2.5)));
}
fields.add(curve);
fields.add(new Field.Curve(UNIT, List.of(new Field.Knot(0.0, new Field.Const(0.0), -3.0), new Field.Knot(1.0, new Field.Const(1.0), -3.0))));
fields.add(new Field.Partition(BASE, List.of(-0.2, 0.4), List.of(new Field.Const(-3.0), new Field.Const(0.5), new Field.Const(7.25))));
fields.add(new Field.Mux(new Field.Binary(Field.BinaryOp.MUL, UNIT, new Field.Const(2.0)), List.of(new Field.Const(4.0), BAND, scaled)));
fields.add(new Field.Gate(BASE, new Field.Binary(Field.BinaryOp.ADD, BAND, new Field.Const(40.0))));
fields.add(new Field.RangeIf(BASE, -0.1, 0.2, new Field.Const(-2.0), BAND));
fields.add(new Field.Grid(new Field.Binary(Field.BinaryOp.ADD, BASE, SPARSE), 4, 8));
fields.add(new Field.Shared("parity/shared", new Field.Binary(Field.BinaryOp.SUB, BAND, BASE)));
fields.add(new Field.Slice(new Field.Binary(Field.BinaryOp.MUL, SPARSE, new Field.Const(-2.0)), Axis.Y, 64));
fields.add(new Field.TopSurface(new Field.Binary(Field.BinaryOp.SUB, BASE, new Field.Const(0.1)), 200, -48, 4));
return fields;
}
private static JsonObject noiseJson(NoiseDef noise) {
JsonArray modifiers = new JsonArray(noise.amplitudes().size());
for (double amplitude : noise.amplitudes()) {
modifiers.add(amplitude);
}
JsonObject json = new JsonObject();
json.addProperty("base_octave", noise.firstOctave());
json.addProperty("octave_count", noise.amplitudes().size());
json.add("amplitude_modifiers", modifiers);
json.addProperty("base_amplitude", 1.0);
json.addProperty("normalize", false);
return json;
}
}
@@ -0,0 +1,242 @@
package art.arcane.iris.probe;
import art.arcane.iris.pack.datapack.DataVersion;
import art.arcane.iris.pack.datapack.export.compile.noise.VanillaNoiseModel;
import net.minecraft.core.HolderGetter;
import net.minecraft.core.registries.Registries;
import net.minecraft.resources.Identifier;
import net.minecraft.resources.ResourceKey;
import net.minecraft.util.RandomSource;
import net.minecraft.world.level.levelgen.NoiseGeneratorSettings;
import net.minecraft.world.level.levelgen.PositionalRandomFactory;
import net.minecraft.world.level.levelgen.RandomState;
import net.minecraft.world.level.levelgen.XoroshiroRandomSource;
import net.minecraft.world.level.levelgen.synth.Noise;
import net.minecraft.world.level.levelgen.synth.NormalNoise;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.ClassRule;
import org.junit.Test;
import java.io.IOException;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.SplittableRandom;
import java.util.stream.Collectors;
import static org.junit.Assert.assertTrue;
public final class VanillaNoiseModelParity263Test {
@ClassRule
public static final PaperRegistryIsolation registryIsolation = new PaperRegistryIsolation();
private static final int POINTS = 4096;
private static final double TOLERANCE = 1.0E-5D;
private static final double RUNTIME_TOLERANCE = 1.0E-4D;
private static final List<Long> WORLD_SEEDS = List.of(0L, 8_675_309L, -4_611_686_018_427_387_904L);
private static final List<ParityNoise> NOISES = List.of(
ParityNoise.mapped("iris_export:parity/single", -7, List.of(1.0D)),
ParityNoise.mapped("iris_export:parity/fbm", -9, List.of(1.0D, 1.0D, 0.5D, 0.25D)),
ParityNoise.mapped("iris_export:parity/sparse", -10, List.of(1.5D, 0.0D, 1.0D, 0.0D, 0.0D, 0.0D)),
ParityNoise.mapped("iris_export:parity/leading_zero", -6, List.of(0.0D, 0.0D, 1.0D, 2.0D)),
ParityNoise.mapped("iris_export:parity/positive", 2, List.of(1.0D, 0.5D)),
ParityNoise.profiled("iris_export:parity/enabled",
"{\"base_octave\":-10,\"octave_count\":6,\"amplitude_modifiers\":[1.5,0.0,1.0,0.0,0.0,0.0],"
+ "\"base_amplitude\":1.2453007926713473}",
new VanillaNoiseModel.OctaveProfile(-10, 6, List.of(1.5D, 0.0D, 1.0D, 0.0D, 0.0D, 0.0D),
1.2453007926713473D, VanillaNoiseModel.Normalization.ENABLED)),
ParityNoise.profiled("iris_export:parity/legacy",
"{\"base_octave\":-9,\"octave_count\":3,\"amplitude_modifiers\":[1.0,0.0,1.0],\"normalize\":\"legacy\"}",
new VanillaNoiseModel.OctaveProfile(-9, 3, List.of(1.0D, 0.0D, 1.0D), 1.0D,
VanillaNoiseModel.Normalization.LEGACY)),
ParityNoise.profiled("iris_export:parity/defaults", "{\"base_octave\":-5}",
new VanillaNoiseModel.OctaveProfile(-5, 1, List.of(), 1.0D, VanillaNoiseModel.Normalization.ENABLED)),
ParityNoise.profiled("iris_export:parity/scaled",
"{\"base_octave\":-6,\"octave_count\":2,\"normalize\":false,\"base_amplitude\":0.5}",
new VanillaNoiseModel.OctaveProfile(-6, 2, List.of(), 0.5D, VanillaNoiseModel.Normalization.DISABLED)));
private static HeadlessNativeRegistries registries;
@BeforeClass
public static void loadNoiseDefinitions() throws IOException {
Map<String, String> definitions = new LinkedHashMap<>();
for (ParityNoise noise : NOISES) {
definitions.put(resourcePath(noise.id()), noise.json());
}
registries = HeadlessNativeRegistries.load(definitions);
}
@AfterClass
public static void closeRegistries() throws IOException {
if (registries != null) {
registries.close();
}
}
@Test
public void modelMatchesNoiseBuiltFromLoadedDefinitions() {
List<String> report = new ArrayList<>();
for (long seed : WORLD_SEEDS) {
PositionalRandomFactory world = new VanillaPrecisionRandom(new XoroshiroRandomSource(seed)).forkPositional();
for (ParityNoise noise : NOISES) {
NormalNoise definition = registries.registries().lookupOrThrow(Registries.NOISE).getOrThrow(key(noise.id())).value();
Noise real = definition.create(world.fromHashOf(Identifier.parse(noise.id())));
report.add(noise.id() + "@" + seed + "=" + assertParity(noise, seed, real, TOLERANCE));
}
}
System.out.println("26.3 vanilla-precision parity " + report);
}
@Test
public void modelMatchesRuntimeRandomStateNoise() {
List<String> report = new ArrayList<>();
for (long seed : WORLD_SEEDS) {
RandomState state = randomState(seed);
for (ParityNoise noise : NOISES) {
Noise real = state.getOrCreateNoise(key(noise.id()));
report.add(noise.id() + "@" + seed + "=" + assertParity(noise, seed, real, RUNTIME_TOLERANCE));
}
}
System.out.println("26.3 runtime parity " + report);
}
private static double assertParity(ParityNoise noise, long seed, Noise real, double tolerance) {
VanillaNoiseModel model = noise.model(seed);
SplittableRandom random = new SplittableRandom(seed ^ noise.id().hashCode());
double worst = 0.0D;
double peak = 0.0D;
for (int index = 0; index < POINTS; index++) {
double range = random.nextInt(4) == 0 ? 30_000_000.0D : 100_000.0D;
double x = random.nextDouble(-range, range);
double y = random.nextDouble(-2_048.0D, 2_048.0D);
double z = random.nextDouble(-range, range);
double expected = real.get(x, y, z);
worst = Math.max(worst, Math.abs(expected - model.sample(x, y, z)));
peak = Math.max(peak, Math.abs(expected));
}
assertTrue(noise.id() + " seed " + seed + " differs by " + worst, worst <= tolerance);
assertTrue(noise.id() + " seed " + seed + " is flat", peak > 0.05D);
return worst;
}
private static RandomState randomState(long seed) {
HolderGetter<NormalNoise> noises = registries.registries().lookupOrThrow(Registries.NOISE);
NoiseGeneratorSettings settings = registries.registries().lookupOrThrow(Registries.NOISE_SETTINGS)
.getOrThrow(NoiseGeneratorSettings.OVERWORLD).value();
return RandomState.create(noises, seed, settings);
}
private static ResourceKey<NormalNoise> key(String id) {
return ResourceKey.create(Registries.NOISE, Identifier.parse(id));
}
private static String resourcePath(String id) {
Identifier identifier = Identifier.parse(id);
return "data/" + identifier.getNamespace() + "/worldgen/noise/" + identifier.getPath() + ".json";
}
private record ParityNoise(String id, String json, int firstOctave, List<Double> amplitudes,
VanillaNoiseModel.OctaveProfile profile) {
private static ParityNoise mapped(String id, int firstOctave, List<Double> amplitudes) {
String modifiers = amplitudes.stream().map(String::valueOf).collect(Collectors.joining(","));
String json = "{\"base_octave\":" + firstOctave + ",\"octave_count\":" + amplitudes.size()
+ ",\"amplitude_modifiers\":[" + modifiers + "],\"base_amplitude\":1.0,\"normalize\":false}";
return new ParityNoise(id, json, firstOctave, amplitudes, null);
}
private static ParityNoise profiled(String id, String json, VanillaNoiseModel.OctaveProfile profile) {
return new ParityNoise(id, json, 0, List.of(), profile);
}
private VanillaNoiseModel model(long seed) {
if (profile == null) {
return VanillaNoiseModel.create(id, firstOctave, amplitudes, seed, DataVersion.V26_3);
}
return VanillaNoiseModel.create(id, profile, seed);
}
}
private static final class VanillaPrecisionRandom implements RandomSource {
private static final double DOUBLE_UNIT = 1.1102230246251565E-16D;
private final RandomSource source;
private VanillaPrecisionRandom(RandomSource source) {
this.source = source;
}
@Override
public RandomSource fork() {
return new VanillaPrecisionRandom(source.fork());
}
@Override
public PositionalRandomFactory forkPositional() {
return new VanillaPrecisionPositional(source.forkPositional());
}
@Override
public void setSeed(long seed) {
source.setSeed(seed);
}
@Override
public int nextInt() {
return source.nextInt();
}
@Override
public int nextInt(int bound) {
return source.nextInt(bound);
}
@Override
public long nextLong() {
return source.nextLong();
}
@Override
public boolean nextBoolean() {
return source.nextBoolean();
}
@Override
public float nextFloat() {
return source.nextFloat();
}
@Override
public double nextDouble() {
return (source.nextLong() >>> 11) * DOUBLE_UNIT;
}
@Override
public double nextGaussian() {
return source.nextGaussian();
}
}
private record VanillaPrecisionPositional(PositionalRandomFactory factory) implements PositionalRandomFactory {
@Override
public RandomSource at(int x, int y, int z) {
return new VanillaPrecisionRandom(factory.at(x, y, z));
}
@Override
public RandomSource fromHashOf(String name) {
return new VanillaPrecisionRandom(factory.fromHashOf(name));
}
@Override
public RandomSource fromSeed(long seed) {
return new VanillaPrecisionRandom(factory.fromSeed(seed));
}
@Override
public void parityConfigString(StringBuilder builder) {
factory.parityConfigString(builder);
}
}
}
@@ -0,0 +1,700 @@
package art.arcane.iris.probe;
import art.arcane.iris.pack.datapack.DataVersion;
import art.arcane.iris.pack.datapack.export.AtomicDatapackPublisher;
import art.arcane.iris.pack.datapack.export.DatapackExportFormat;
import art.arcane.iris.pack.datapack.export.ExportIrFixtures;
import art.arcane.iris.pack.datapack.export.GenerationCost;
import art.arcane.iris.pack.datapack.export.WorldgenDatapackExporter;
import art.arcane.iris.pack.datapack.export.WorldgenExportMode;
import art.arcane.iris.pack.datapack.export.WorldgenExportProgress;
import art.arcane.iris.pack.datapack.export.WorldgenExportRequest;
import art.arcane.iris.pack.datapack.export.WorldgenExportResult;
import art.arcane.iris.pack.datapack.export.compile.PackCompiler;
import art.arcane.iris.pack.datapack.export.compile.surface.MaterialProgramCompiler;
import art.arcane.iris.pack.datapack.export.compile.terrain.FieldMacros;
import art.arcane.iris.pack.datapack.export.compile.terrain.HeightCompiler;
import art.arcane.iris.pack.datapack.export.emit.NestingSplitter;
import art.arcane.iris.pack.datapack.export.emit.WorldgenJsonWriter;
import art.arcane.iris.pack.datapack.export.ir.ExportDiagnostic;
import art.arcane.iris.pack.datapack.export.ir.ExportIr;
import art.arcane.iris.pack.datapack.export.ir.ExportLimit;
import art.arcane.iris.pack.datapack.export.ir.Field;
import art.arcane.iris.pack.datapack.export.ir.FieldEvaluator;
import art.arcane.iris.pack.datapack.export.ir.NoiseDef;
import art.arcane.iris.pack.datapack.export.lower.GenerationBudget;
import com.google.gson.JsonObject;
import com.google.gson.JsonParser;
import net.minecraft.core.BlockPos;
import net.minecraft.core.Holder;
import net.minecraft.core.HolderSet;
import net.minecraft.core.registries.Registries;
import net.minecraft.resources.Identifier;
import net.minecraft.resources.ResourceKey;
import net.minecraft.tags.WorldPresetTags;
import net.minecraft.util.Interval;
import net.minecraft.world.level.ChunkPos;
import net.minecraft.world.level.biome.Biome;
import net.minecraft.world.level.block.Blocks;
import net.minecraft.world.level.block.state.BlockState;
import net.minecraft.world.level.chunk.ProtoChunk;
import net.minecraft.world.level.dimension.LevelStem;
import net.minecraft.world.level.levelgen.densityfunction.DensityFunction;
import net.minecraft.world.level.levelgen.densityfunction.DfRewriteRule;
import net.minecraft.world.level.levelgen.densityfunction.op.BinaryFunction;
import net.minecraft.world.level.levelgen.material.rule.OreVeinRule;
import net.minecraft.world.level.levelgen.presets.WorldPreset;
import net.minecraft.world.level.levelgen.structure.BoundingBox;
import net.minecraft.world.level.levelgen.structure.Structure;
import net.minecraft.world.level.levelgen.structure.StructureStart;
import org.junit.AfterClass;
import org.junit.Assume;
import org.junit.BeforeClass;
import org.junit.ClassRule;
import org.junit.Test;
import org.junit.rules.TemporaryFolder;
import java.io.ByteArrayInputStream;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayDeque;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Deque;
import java.util.Enumeration;
import java.util.EnumMap;
import java.util.HashMap;
import java.util.HashSet;
import java.util.IdentityHashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Set;
import java.util.SortedMap;
import java.util.TreeMap;
import java.util.TreeSet;
import java.util.concurrent.atomic.AtomicReference;
import java.util.zip.GZIPInputStream;
import java.util.zip.ZipEntry;
import java.util.zip.ZipFile;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;
public final class WorldgenExport263CodecTest {
@ClassRule
public static final PaperRegistryIsolation registryIsolation = new PaperRegistryIsolation();
@ClassRule
public static final TemporaryFolder overworldFolder = new TemporaryFolder();
private static final long SEED = 1337L;
private static final String DIMENSION_ID = ExportIrFixtures.NAMESPACE + ":" + ExportIrFixtures.DIMENSION;
private static final String HEIGHT_ID = ExportIrFixtures.NAMESPACE + ":" + ExportIrFixtures.DIMENSION + "/height";
private static final double SURFACE_TOLERANCE = 2.0;
private static final String BIOME_PREFIX = "data/" + ExportIrFixtures.NAMESPACE + "/worldgen/biome/";
private static final int[] SAMPLE_Y = {-64, 0, 64, 200};
private static final int LATTICE = 16;
private static final int LATTICE_STEP = 61;
private static final int HEIGHT_POINTS = 8;
private static final int HEIGHT_STEP = 97;
private static final String PROBE_PACK = "iris.exportProbePack";
private static final String OVERWORLD = "overworld";
private static final String OVERWORLD_ID = WorldgenExportRequest.DEFAULT_NAMESPACE + ":" + OVERWORLD;
private static final String OVERWORLD_BIOME_PREFIX = "data/" + WorldgenExportRequest.DEFAULT_NAMESPACE + "/worldgen/biome/";
private static final String LOG_PREFIX = "[worldgen-export]";
private static final int OVERWORLD_LATTICE = 4;
private static final int OVERWORLD_STEP = 509;
private static final int MAX_NESTING = 255;
private static final String CAVE_FIELD = "cave/field";
private static final String WALL_FALLBACK = "uses the nearest kept palette (minecraft:";
private static final double WALL_SHELL_BAND_LIMIT = 0.25;
private static final String DENSITY_FUNCTION_RESOURCE = "worldgen/density_function";
private static final double MEGABYTE = 1024.0 * 1024.0;
private static final int OVERWORLD_LAVA_Y = -248;
private static final int LAVA_LINE = -54;
private static final int DEEP_CHUNKS = 2;
private static final int DEEP_STRIDE = 4;
private static final int CHUNK_WIDTH = 16;
private static final float OPEN_MARGIN = 1.0e-3f;
private static final int LAVA_SEARCH_RADIUS = 1024;
private static final int LAVA_SEARCH_STEP = 16;
private static final int LAVA_CELLS = 4;
private static final int AGREEMENT_POINTS = 16;
private static final double AGREEMENT_TOLERANCE = 1.0;
private static final int AGREEMENT_OUTLIERS = 2;
private static final String SUBTERRAIN_STRUCTURES = "data/" + WorldgenExportRequest.DEFAULT_NAMESPACE + "/worldgen/structure/subterrain/";
private static final String SUBTERRAIN_TEMPLATES = "data/" + WorldgenExportRequest.DEFAULT_NAMESPACE + "/structure/subterrain/";
private static final int ROOM_GRID = 24;
private static final int ROOM_STEP_CHUNKS = 8;
private static final int ROOM_ATTEMPTS = 4;
private static final int ROOM_REACH = 128;
private static final Map<WorldgenExportMode, SortedMap<String, byte[]>> FILES = new EnumMap<>(WorldgenExportMode.class);
private static final Map<WorldgenExportMode, WorldgenExportSampler> SAMPLERS = new EnumMap<>(WorldgenExportMode.class);
private static SortedMap<String, byte[]> exportedOverworld;
private static WorldgenExportResult exportedOverworldResult;
private static ExportIr exportedOverworldIr;
@BeforeClass
public static void loadExports() throws IOException {
for (WorldgenExportMode mode : WorldgenExportMode.values()) {
SortedMap<String, byte[]> files = WorldgenJsonWriter.forTarget(DataVersion.V26_3).write(ExportIrFixtures.flatNoise(DataVersion.V26_3, mode));
FILES.put(mode, files);
SAMPLERS.put(mode, WorldgenExportSampler.open(files, mode, DIMENSION_ID, SEED));
}
}
@AfterClass
public static void closeExports() throws IOException {
for (WorldgenExportSampler sampler : SAMPLERS.values()) {
sampler.close();
}
SAMPLERS.clear();
FILES.clear();
exportedOverworld = null;
exportedOverworldResult = null;
exportedOverworldIr = null;
}
@Test
public void fixtureLoadsInEveryMode() {
for (WorldgenExportMode mode : WorldgenExportMode.values()) {
WorldgenExportSampler sampler = SAMPLERS.get(mode);
assertEquals(ExportIrFixtures.MIN_Y, sampler.minY());
assertEquals(ExportIrFixtures.HEIGHT, sampler.height());
assertEquals(ExportIrFixtures.SEA_LEVEL, sampler.seaLevel());
assertFalse(sampler.densityFunctionIds().isEmpty());
for (String id : sampler.densityFunctionIds()) {
assertTrue(mode + " " + id, sampler.registries().lookupOrThrow(Registries.DENSITY_FUNCTION).containsKey(Identifier.parse(id)));
}
assertTrue(sampler.registries().lookupOrThrow(Registries.NOISE_SETTINGS).containsKey(Identifier.parse(DIMENSION_ID)));
assertTrue(sampler.registries().lookupOrThrow(Registries.DIMENSION_TYPE).containsKey(Identifier.parse(DIMENSION_ID)));
assertTrue(sampler.registries().lookupOrThrow(Registries.MATERIAL_RULE).containsKey(Identifier.parse(DIMENSION_ID)));
boolean preset = sampler.registries().lookupOrThrow(Registries.WORLD_PRESET).containsKey(Identifier.parse(DIMENSION_ID));
boolean dimension = sampler.registries().lookupOrThrow(Registries.LEVEL_STEM).containsKey(Identifier.parse(DIMENSION_ID));
boolean overworld = sampler.registries().lookupOrThrow(Registries.LEVEL_STEM).containsKey(LevelStem.OVERWORLD.identifier());
assertEquals(mode.toString(), mode == WorldgenExportMode.PRESET, preset);
assertEquals(mode.toString(), mode == WorldgenExportMode.DIMENSION, dimension);
assertEquals(mode.toString(), mode == WorldgenExportMode.OVERWORLD, overworld);
}
WorldgenExportSampler presetSampler = SAMPLERS.get(WorldgenExportMode.PRESET);
Holder<WorldPreset> preset = presetSampler.registries().lookupOrThrow(Registries.WORLD_PRESET)
.getOrThrow(ResourceKey.create(Registries.WORLD_PRESET, Identifier.parse(DIMENSION_ID)));
assertTrue(preset.value().createWorldDimensions().get(LevelStem.NETHER).isPresent());
assertTrue(preset.value().createWorldDimensions().get(LevelStem.END).isPresent());
HolderSet.Named<WorldPreset> normal = presetSampler.registries().lookupOrThrow(Registries.WORLD_PRESET).getOrThrow(WorldPresetTags.NORMAL);
assertTrue(normal.contains(preset));
}
@Test
public void everyDensityFunctionEvaluatesFinite() {
for (WorldgenExportMode mode : WorldgenExportMode.values()) {
WorldgenExportSampler sampler = SAMPLERS.get(mode);
for (String id : sampler.densityFunctionIds()) {
for (int y : SAMPLE_Y) {
for (int xIndex = 0; xIndex < LATTICE; xIndex++) {
for (int zIndex = 0; zIndex < LATTICE; zIndex++) {
int x = (xIndex - LATTICE / 2) * LATTICE_STEP;
int z = (zIndex - LATTICE / 2) * LATTICE_STEP;
float value = sampler.density(id, x, y, z);
assertTrue(mode + " " + id + " at " + x + "," + y + "," + z + " = " + value, Float.isFinite(value));
}
}
}
}
}
}
@Test
public void baseHeightWithinDimension() {
for (WorldgenExportMode mode : WorldgenExportMode.values()) {
WorldgenExportSampler sampler = SAMPLERS.get(mode);
Set<Integer> heights = new HashSet<>();
int land = 0;
for (int xIndex = 0; xIndex < HEIGHT_POINTS; xIndex++) {
for (int zIndex = 0; zIndex < HEIGHT_POINTS; zIndex++) {
int x = (xIndex - HEIGHT_POINTS / 2) * HEIGHT_STEP;
int z = (zIndex - HEIGHT_POINTS / 2) * HEIGHT_STEP;
int height = sampler.baseHeight(x, z);
assertTrue(mode + " height " + height + " at " + x + "," + z,
height >= sampler.minY() && height <= sampler.minY() + sampler.height());
float surface = sampler.density(HEIGHT_ID, x, 0, z);
int floor = sampler.floorHeight(x, z);
assertTrue(mode + " height " + height + " below floor " + floor, height >= floor);
if (surface > sampler.seaLevel() + 1) {
assertTrue(mode + " floor " + floor + " against surface " + surface, Math.abs(floor - surface) <= SURFACE_TOLERANCE);
land++;
}
heights.add(height);
}
}
assertTrue(mode + " surface never varies", heights.size() > 1);
assertTrue(mode + " has no land column above sea level", land > 0);
}
}
@Test
public void resolverReturnsOnlyExportedBiomes() {
for (WorldgenExportMode mode : WorldgenExportMode.values()) {
WorldgenExportSampler sampler = SAMPLERS.get(mode);
Set<String> exported = exportedBiomes(FILES.get(mode), BIOME_PREFIX, ExportIrFixtures.NAMESPACE);
assertFalse(exported.isEmpty());
assertTrue(mode + " " + sampler.possibleBiomes(), exported.containsAll(sampler.possibleBiomes()));
for (int y : SAMPLE_Y) {
for (int xIndex = 0; xIndex < LATTICE; xIndex++) {
for (int zIndex = 0; zIndex < LATTICE; zIndex++) {
String biome = sampler.biome((xIndex - LATTICE / 2) * LATTICE_STEP, y, (zIndex - LATTICE / 2) * LATTICE_STEP);
assertTrue(mode + " " + biome, exported.contains(biome));
}
}
}
}
}
@Test
public void overworldPackLoads() throws Exception {
SortedMap<String, byte[]> files = overworldFiles();
assertGenerationBudget(exportedOverworldResult);
assertWallPalettesFallBack(exportedOverworldResult);
for (Map.Entry<String, byte[]> file : files.entrySet()) {
if (file.getKey().endsWith(".json")) {
int depth = NestingSplitter.depth(JsonParser.parseString(new String(file.getValue(), StandardCharsets.UTF_8)));
assertTrue(file.getKey() + " nests " + depth, depth <= MAX_NESTING);
}
if (file.getKey().endsWith(".nbt")) {
assertCompleteGzip(file.getKey(), file.getValue());
}
}
try (WorldgenExportSampler sampler = WorldgenExportSampler.open(files, WorldgenExportMode.OVERWORLD, OVERWORLD_ID, SEED)) {
Set<String> roots = new TreeSet<>();
for (String id : sampler.densityFunctionIds()) {
if (id.startsWith(OVERWORLD_ID + "/")) {
roots.add(id);
}
}
assertTrue(roots.toString(), roots.contains(OVERWORLD_ID + "/final_density"));
for (String id : roots) {
for (int y : SAMPLE_Y) {
for (int xIndex = 0; xIndex < OVERWORLD_LATTICE; xIndex++) {
for (int zIndex = 0; zIndex < OVERWORLD_LATTICE; zIndex++) {
int x = (xIndex - OVERWORLD_LATTICE / 2) * OVERWORLD_STEP;
int z = (zIndex - OVERWORLD_LATTICE / 2) * OVERWORLD_STEP;
float value = sampler.density(id, x, y, z);
assertTrue(id + " at " + x + "," + y + "," + z + " = " + value, Float.isFinite(value));
}
}
}
}
assertNaturalHeightMatchesTheEvaluator(exportedOverworldIr, sampler);
Set<String> exported = exportedBiomes(files, OVERWORLD_BIOME_PREFIX, WorldgenExportRequest.DEFAULT_NAMESPACE);
assertFalse(exported.isEmpty());
assertTrue(sampler.possibleBiomes().toString(), exported.containsAll(sampler.possibleBiomes()));
for (int xIndex = 0; xIndex < HEIGHT_POINTS; xIndex++) {
for (int zIndex = 0; zIndex < HEIGHT_POINTS; zIndex++) {
int x = (xIndex - HEIGHT_POINTS / 2) * HEIGHT_STEP;
int z = (zIndex - HEIGHT_POINTS / 2) * HEIGHT_STEP;
int height = sampler.baseHeight(x, z);
assertTrue("height " + height + " at " + x + "," + z, height >= sampler.minY() && height <= sampler.minY() + sampler.height());
for (int y : SAMPLE_Y) {
String biome = sampler.biome(x, y, z);
assertTrue(biome + " at " + x + "," + y + "," + z, exported.contains(biome));
}
}
}
}
}
@Test
public void overworldHasNoDisjointMinMax() throws Exception {
SortedMap<String, byte[]> files = overworldFiles();
try (WorldgenExportSampler sampler = WorldgenExportSampler.open(files, WorldgenExportMode.OVERWORLD, OVERWORLD_ID, SEED)) {
MinMaxAudit audit = new MinMaxAudit();
for (String id : sampler.densityFunctionIds()) {
audit.rewrite(sampler.registries().lookupOrThrow(Registries.DENSITY_FUNCTION)
.getOrThrow(ResourceKey.create(Registries.DENSITY_FUNCTION, Identifier.parse(id))).value());
}
System.out.println(LOG_PREFIX + " min/max nodes " + audit.checked() + ", disjoint " + audit.disjoint().size());
assertTrue(audit.checked() > 0);
assertTrue(audit.disjoint().size() + " disjoint min/max nodes: " + audit.disjoint(), audit.disjoint().isEmpty());
}
}
@Test
public void overworldDeepCavesOpenWithoutLava() throws Exception {
SortedMap<String, byte[]> files = overworldFiles();
String field = WorldgenExportRequest.DEFAULT_NAMESPACE + ":" + CAVE_FIELD;
long open = 0L;
long lava = 0L;
try (WorldgenExportSampler sampler = WorldgenExportSampler.open(files, WorldgenExportMode.OVERWORLD, OVERWORLD_ID, SEED)) {
assertTrue(field, sampler.densityFunctionIds().contains(field));
BlockPos.MutableBlockPos position = new BlockPos.MutableBlockPos();
for (int chunkX = -DEEP_CHUNKS; chunkX < DEEP_CHUNKS; chunkX++) {
for (int chunkZ = -DEEP_CHUNKS; chunkZ < DEEP_CHUNKS; chunkZ++) {
ProtoChunk chunk = sampler.terrainChunk(chunkX, chunkZ);
for (int x = 0; x < CHUNK_WIDTH; x++) {
for (int z = 0; z < CHUNK_WIDTH; z++) {
for (int y = OVERWORLD_LAVA_Y + 1; y < LAVA_LINE; y++) {
BlockState state = chunk.getBlockState(position.set(x, y, z));
assertFalse(state + " at chunk " + chunkX + "," + chunkZ + " " + x + "," + y + "," + z, state.is(Blocks.LAVA));
}
}
}
for (int x = 0; x < CHUNK_WIDTH; x += DEEP_STRIDE) {
for (int z = 0; z < CHUNK_WIDTH; z += DEEP_STRIDE) {
int blockX = (chunkX << 4) + x;
int blockZ = (chunkZ << 4) + z;
for (int y = OVERWORLD_LAVA_Y + 1; y < LAVA_LINE; y++) {
if (sampler.density(field, blockX, y, blockZ) > -OPEN_MARGIN) {
continue;
}
BlockState state = chunk.getBlockState(position.set(x, y, z));
assertTrue(state + " at " + blockX + "," + y + "," + blockZ, state.isAir());
open++;
}
}
}
}
}
for (BlockPos cell : openLavaBandCells(sampler, field)) {
ProtoChunk chunk = sampler.terrainChunk(cell.getX() >> 4, cell.getZ() >> 4);
BlockState state = chunk.getBlockState(position.set(cell.getX() & (CHUNK_WIDTH - 1), cell.getY(), cell.getZ() & (CHUNK_WIDTH - 1)));
assertTrue(state + " at " + cell, state.is(Blocks.LAVA));
lava++;
}
}
System.out.println(LOG_PREFIX + " deep cave cells open " + open + ", lava band cells " + lava);
assertTrue("no deep cave cell opened", open > 0L);
assertTrue("no lava band cell opened", lava > 0L);
}
@Test
public void overworldMaterialPassComputesTheCaveFieldOnce() throws Exception {
SortedMap<String, byte[]> files = overworldFiles();
try (WorldgenExportSampler sampler = WorldgenExportSampler.open(files, WorldgenExportMode.OVERWORLD, OVERWORLD_ID, SEED)) {
CompiledCaches caches = CompiledCaches.open(sampler.registries(), SEED);
int field = caches.rootId(caches.function(WorldgenExportRequest.DEFAULT_NAMESPACE + ":" + CAVE_FIELD));
List<OreVeinRule> veins = caches.oreVeins();
DensityFunction opening = veins.getFirst().density();
assertTrue(caches.ids(opening).contains(field));
assertTrue(caches.ids(caches.finalDensity()).contains(field));
Set<Integer> caveCaches = caches.ids(caches.function(WorldgenExportRequest.DEFAULT_NAMESPACE + ":" + CAVE_FIELD));
int shells = 0;
for (OreVeinRule vein : veins) {
if (!vein.oreBlock().isAir()) {
assertTrue("wall shell " + shells + " " + vein.oreBlock(), caches.ids(vein.density()).contains(field));
assertTrue("wall shell " + shells + " " + vein.oreBlock(), caveCaches.containsAll(caches.ids(vein.density())));
shells++;
}
}
JsonObject shell = JsonParser.parseString(new String(files.get("data/" + WorldgenExportRequest.DEFAULT_NAMESPACE + "/" + DENSITY_FUNCTION_RESOURCE
+ "/" + MaterialProgramCompiler.WALL_SHELL + ".json"), StandardCharsets.UTF_8)).getAsJsonObject();
double band = shell.get("max_exclusive").getAsDouble();
System.out.println(LOG_PREFIX + " material pass: " + veins.size() + " ore vein rules, " + shells + " wall shells read cave field cache " + field
+ " within band " + band);
assertTrue(String.valueOf(shells), shells > 0);
assertTrue(String.valueOf(band), band > 0.0 && band < WALL_SHELL_BAND_LIMIT);
}
}
@Test
public void overworldSubterrainRoomsPlace() throws Exception {
SortedMap<String, byte[]> files = overworldFiles();
SortedMap<String, JsonObject> structures = new TreeMap<>();
for (Map.Entry<String, byte[]> file : files.entrySet()) {
if (file.getKey().startsWith(SUBTERRAIN_STRUCTURES) && file.getKey().endsWith(".json")) {
String path = file.getKey().substring(SUBTERRAIN_STRUCTURES.length(), file.getKey().length() - ".json".length());
structures.put(WorldgenExportRequest.DEFAULT_NAMESPACE + ":subterrain/" + path,
JsonParser.parseString(new String(file.getValue(), StandardCharsets.UTF_8)).getAsJsonObject());
}
}
long templateBytes = 0L;
for (Map.Entry<String, byte[]> file : files.entrySet()) {
if (file.getKey().startsWith(SUBTERRAIN_TEMPLATES)) {
templateBytes += file.getValue().length;
}
}
System.out.println(LOG_PREFIX + " subterrain structures " + structures.size() + ", structure sets "
+ count(files, "/worldgen/structure_set/subterrain/", ".json") + ", templates " + count(files, "/structure/subterrain/", ".nbt")
+ ", template bytes " + templateBytes);
assertFalse(structures.isEmpty());
try (WorldgenExportSampler sampler = WorldgenExportSampler.open(files, WorldgenExportMode.OVERWORLD, OVERWORLD_ID, SEED)) {
for (Map.Entry<String, JsonObject> entry : structures.entrySet()) {
Structure structure = sampler.registries().lookupOrThrow(Registries.STRUCTURE)
.getOrThrow(ResourceKey.create(Registries.STRUCTURE, Identifier.parse(entry.getKey()))).value();
JsonObject height = entry.getValue().getAsJsonObject("start_height");
int y = (height.getAsJsonObject("min_inclusive").get("absolute").getAsInt()
+ height.getAsJsonObject("max_inclusive").get("absolute").getAsInt()) / 2;
List<ChunkPos> matches = new ArrayList<>();
for (int xIndex = 0; xIndex < ROOM_GRID; xIndex++) {
for (int zIndex = 0; zIndex < ROOM_GRID; zIndex++) {
ChunkPos chunk = new ChunkPos((xIndex - ROOM_GRID / 2) * ROOM_STEP_CHUNKS, (zIndex - ROOM_GRID / 2) * ROOM_STEP_CHUNKS);
String biome = sampler.biome(chunk.getMinBlockX(), y, chunk.getMinBlockZ());
Holder<Biome> holder = sampler.registries().lookupOrThrow(Registries.BIOME)
.getOrThrow(ResourceKey.create(Registries.BIOME, Identifier.parse(biome)));
if (structure.biomes().contains(holder)) {
matches.add(chunk);
}
}
}
System.out.println(String.format(Locale.ROOT, "%s subterrain %s biome match %d/%d at y %d", LOG_PREFIX, entry.getKey(),
matches.size(), ROOM_GRID * ROOM_GRID, y));
assertFalse(entry.getKey() + " finds no biome at y " + y, matches.isEmpty());
StructureStart start = null;
for (ChunkPos chunk : matches.subList(0, Math.min(ROOM_ATTEMPTS, matches.size()))) {
StructureStart candidate = sampler.structureStart(entry.getKey(), chunk.x(), chunk.z());
if (candidate.isValid()) {
start = candidate;
BoundingBox box = start.getBoundingBox();
assertTrue(entry.getKey() + " " + box, chunk.getMinBlockX() - box.minX() <= ROOM_REACH && box.maxX() - chunk.getMinBlockX() <= ROOM_REACH);
assertTrue(entry.getKey() + " " + box, chunk.getMinBlockZ() - box.minZ() <= ROOM_REACH && box.maxZ() - chunk.getMinBlockZ() <= ROOM_REACH);
assertTrue(entry.getKey() + " " + box, box.minY() >= sampler.minY() && box.maxY() < sampler.minY() + sampler.height());
break;
}
}
assertTrue(entry.getKey() + " never starts", start != null);
}
}
}
private static List<BlockPos> openLavaBandCells(WorldgenExportSampler sampler, String field) {
List<BlockPos> cells = new ArrayList<>(LAVA_CELLS);
for (int x = -LAVA_SEARCH_RADIUS; x < LAVA_SEARCH_RADIUS; x += LAVA_SEARCH_STEP) {
for (int z = -LAVA_SEARCH_RADIUS; z < LAVA_SEARCH_RADIUS; z += LAVA_SEARCH_STEP) {
for (int y = sampler.minY(); y <= OVERWORLD_LAVA_Y; y++) {
if (sampler.density(field, x, y, z) > -OPEN_MARGIN) {
continue;
}
cells.add(new BlockPos(x, y, z));
if (cells.size() == LAVA_CELLS) {
return cells;
}
}
}
}
return cells;
}
private static synchronized SortedMap<String, byte[]> overworldFiles() throws Exception {
String probePack = System.getProperty(PROBE_PACK);
Assume.assumeTrue("Set -P" + PROBE_PACK + " to export a real pack", probePack != null && !probePack.isBlank());
if (exportedOverworld != null) {
return exportedOverworld;
}
Path root = overworldFolder.newFolder("exports").toPath();
SortedMap<String, byte[]> untrimmed = new TreeMap<>();
AtomicReference<ExportIr> compiled = new AtomicReference<>();
WorldgenExportResult result;
try (RealPackProbeSupport.Workspace workspace = RealPackProbeSupport.openWorkspace(new RealPackProbeSupport.WorkspaceOptions(
new File(probePack), OVERWORLD, LOG_PREFIX, new HeadlessNativeRuntime(), overworldFolder.newFolder("workspace").toPath()))) {
WorldgenExportRequest request = WorldgenExportRequest.builder()
.pack(workspace.pack().toPath())
.dimension(OVERWORLD)
.exportRoot(root)
.objects(true)
.build();
result = new WorldgenDatapackExporter(target -> ir -> {
compiled.set(ir);
SortedMap<String, byte[]> written = WorldgenJsonWriter.forTarget(target).write(ir);
untrimmed.putAll(written);
return written;
}, PackCompiler.create()).export(request, WorldgenExportProgress.NONE);
}
assertTrue(result.message() + " " + result.diagnostics(), result.isSuccess());
SortedMap<String, byte[]> files = zipEntries(result.output());
Path untrimmedZip = root.resolve("untrimmed.zip");
new AtomicDatapackPublisher().publish(untrimmed, untrimmedZip, DatapackExportFormat.ZIP, false, ".untrimmed-");
report(result, untrimmed, Files.size(untrimmedZip), files);
exportedOverworld = files;
exportedOverworldResult = result;
exportedOverworldIr = compiled.get();
return files;
}
private static void assertGenerationBudget(WorldgenExportResult result) {
GenerationCost cost = result.cost();
Set<String> flagged = new TreeSet<>();
for (ExportDiagnostic diagnostic : result.diagnostics()) {
if (diagnostic.limit() == ExportLimit.GENERATION_COST) {
flagged.add(diagnostic.resource());
}
}
for (GenerationCost.Root root : GenerationCost.Root.values()) {
GenerationCost.RootCost rootCost = cost.root(root).orElseThrow();
System.out.println(String.format(Locale.ROOT, "%s cost %s worst case %.3e expanded %.3e distinct %d", LOG_PREFIX, root.id(),
rootCost.worstCaseNodes(), rootCost.expandedNodes(), rootCost.distinctNodes()));
}
for (GenerationCost.Root root : GenerationCost.Root.values()) {
if (!root.climate()) {
continue;
}
double limit = GenerationBudget.worstCaseLimit(root).orElseThrow();
double nodes = cost.root(root).orElseThrow().worstCaseNodes();
String resource = "noise_settings/" + OVERWORLD + "/" + root.id();
assertTrue(root.id() + " evaluates " + nodes + " nodes, budget " + limit, nodes <= limit);
assertFalse(resource + " " + flagged, flagged.contains(resource));
}
long finalDensity = cost.root(GenerationCost.Root.FINAL_DENSITY).orElseThrow().distinctNodes();
String finalResource = "noise_settings/" + OVERWORLD + "/" + GenerationCost.Root.FINAL_DENSITY.id();
System.out.println(String.format(Locale.ROOT, "%s final_density distinct %d of %d, density function JSON %.1f of %.1f MB", LOG_PREFIX, finalDensity,
GenerationBudget.FINAL_DENSITY_DISTINCT_NODES, cost.densityFunctionBytes() / MEGABYTE, GenerationBudget.DENSITY_FUNCTION_JSON_BYTES / MEGABYTE));
assertEquals(finalResource + " " + flagged, finalDensity > GenerationBudget.FINAL_DENSITY_DISTINCT_NODES, flagged.contains(finalResource));
assertTrue(cost.densityFunctionBytes() + " bytes", cost.densityFunctionBytes() > 0L);
assertTrue(cost.densityFunctionBytes() + " bytes", cost.densityFunctionBytes() <= GenerationBudget.DENSITY_FUNCTION_JSON_BYTES);
assertFalse(flagged.toString(), flagged.contains(DENSITY_FUNCTION_RESOURCE));
}
private static void assertWallPalettesFallBack(WorldgenExportResult result) {
int fallbacks = 0;
for (ExportDiagnostic diagnostic : result.diagnostics()) {
if (diagnostic.limit() == ExportLimit.WALL_PALETTE && diagnostic.severity() == ExportDiagnostic.Severity.WARNING) {
assertTrue(diagnostic.resource() + " " + diagnostic.message(), diagnostic.message().contains(WALL_FALLBACK));
fallbacks++;
}
}
System.out.println(LOG_PREFIX + " wall palettes over the ore vein budget " + fallbacks);
}
private static void assertNaturalHeightMatchesTheEvaluator(ExportIr ir, WorldgenExportSampler sampler) {
Field.Shared height = shared(ir.router().chunkSurfaceLevel(), HeightCompiler.HEIGHT);
Map<String, NoiseDef> noises = new HashMap<>();
for (NoiseDef noise : ir.noises()) {
noises.put(ir.namespace() + ":" + noise.id(), noise);
}
String heightId = ir.namespace() + ":" + HeightCompiler.HEIGHT;
int outliers = 0;
StringBuilder detail = new StringBuilder();
for (int xIndex = 0; xIndex < AGREEMENT_POINTS; xIndex++) {
for (int zIndex = 0; zIndex < AGREEMENT_POINTS; zIndex++) {
int x = (xIndex - AGREEMENT_POINTS / 2) * OVERWORLD_STEP;
int z = (zIndex - AGREEMENT_POINTS / 2) * OVERWORLD_STEP;
double evaluated = FieldEvaluator.sample(height.expr(), noises, SEED, DataVersion.V26_3, x, 0, z);
double exported = sampler.density(heightId, x, 0, z);
if (Math.abs(evaluated - exported) > AGREEMENT_TOLERANCE) {
outliers++;
detail.append(' ').append(x).append(',').append(z).append('=').append(evaluated).append('/').append(exported);
}
}
}
assertTrue(outliers + " columns differ:" + detail, outliers <= AGREEMENT_OUTLIERS);
}
private static Field.Shared shared(Field root, String name) {
Set<Field> visited = Collections.newSetFromMap(new IdentityHashMap<>());
Deque<Field> pending = new ArrayDeque<>();
pending.push(root);
while (!pending.isEmpty()) {
Field field = pending.pop();
if (!visited.add(field)) {
continue;
}
if (field instanceof Field.Shared candidate && candidate.name().equals(name)) {
return candidate;
}
for (Field child : FieldMacros.children(field)) {
pending.push(child);
}
}
throw new AssertionError("Router has no shared field " + name);
}
private static void assertCompleteGzip(String path, byte[] bytes) throws IOException {
try (InputStream input = new GZIPInputStream(new ByteArrayInputStream(bytes))) {
input.readAllBytes();
assertEquals(path, -1, input.read());
}
}
private static void report(WorldgenExportResult result, SortedMap<String, byte[]> untrimmed, long untrimmedZipBytes,
SortedMap<String, byte[]> files) throws IOException {
SortedMap<ExportLimit, Integer> limits = new TreeMap<>();
for (ExportDiagnostic diagnostic : result.diagnostics()) {
limits.merge(diagnostic.limit(), 1, Integer::sum);
}
System.out.println(LOG_PREFIX + " untrimmed zip bytes " + untrimmedZipBytes + ", files " + untrimmed.size()
+ ", trimmed zip bytes " + Files.size(result.output()) + ", files " + files.size() + ", trim " + result.trim());
System.out.println(LOG_PREFIX + " diagnostics " + result.diagnostics().size() + " " + limits);
reportCounts("untrimmed", untrimmed);
reportCounts("trimmed", files);
}
private static void reportCounts(String label, SortedMap<String, byte[]> files) {
System.out.println(LOG_PREFIX + " " + label + " templates " + count(files, "/structure/", ".nbt")
+ ", structures " + count(files, "/worldgen/structure/", ".json")
+ ", structure sets " + count(files, "/worldgen/structure_set/", ".json")
+ ", placed features " + count(files, "/worldgen/placed_feature/", ".json")
+ ", features " + count(files, "/worldgen/feature/", ".json")
+ ", density functions " + count(files, "/worldgen/density_function/", ".json")
+ ", noises " + count(files, "/worldgen/noise/", ".json")
+ ", biomes " + count(files, "/worldgen/biome/", ".json"));
}
private static int count(SortedMap<String, byte[]> files, String folder, String suffix) {
int count = 0;
for (String path : files.keySet()) {
if (path.contains(folder) && path.endsWith(suffix)) {
count++;
}
}
return count;
}
private static SortedMap<String, byte[]> zipEntries(Path zip) throws IOException {
SortedMap<String, byte[]> files = new TreeMap<>();
try (ZipFile archive = new ZipFile(zip.toFile())) {
Enumeration<? extends ZipEntry> entries = archive.entries();
while (entries.hasMoreElements()) {
ZipEntry entry = entries.nextElement();
if (!entry.isDirectory()) {
files.put(entry.getName(), archive.getInputStream(entry).readAllBytes());
}
}
}
return files;
}
private static Set<String> exportedBiomes(SortedMap<String, byte[]> files, String prefix, String namespace) {
Set<String> biomes = new TreeSet<>();
for (String path : files.keySet()) {
if (path.startsWith(prefix)) {
biomes.add(namespace + ":" + path.substring(prefix.length(), path.length() - ".json".length()));
}
}
return biomes;
}
private static final class MinMaxAudit implements DfRewriteRule {
private final Set<DensityFunction> visited = Collections.newSetFromMap(new IdentityHashMap<>());
private final List<String> disjoint = new ArrayList<>();
private int checked;
@Override
public DensityFunction rewrite(DensityFunction function) {
if (!visited.add(function)) {
return function;
}
if (function instanceof BinaryFunction binary && (binary.type() == BinaryFunction.Type.MIN || binary.type() == BinaryFunction.Type.MAX)) {
checked++;
Interval left = binary.left().range();
Interval right = binary.right().range();
if (left.max() < right.min() || right.max() < left.min()) {
disjoint.add(binary.type() + " " + left + " " + right);
}
}
function.rewriteChildren(this);
return function;
}
private int checked() {
return checked;
}
private List<String> disjoint() {
return disjoint;
}
}
}
@@ -0,0 +1,136 @@
package art.arcane.iris.probe;
import art.arcane.iris.pack.datapack.export.ExportTestPacks;
import art.arcane.iris.pack.datapack.export.WorldgenDatapackExporter;
import art.arcane.iris.pack.datapack.export.WorldgenExportMode;
import art.arcane.iris.pack.datapack.export.WorldgenExportProgress;
import art.arcane.iris.pack.datapack.export.WorldgenExportRequest;
import art.arcane.iris.pack.datapack.export.WorldgenExportResult;
import com.google.gson.JsonObject;
import com.google.gson.JsonParser;
import net.minecraft.core.BlockPos;
import net.minecraft.world.level.block.Blocks;
import net.minecraft.world.level.block.state.BlockState;
import net.minecraft.world.level.chunk.ProtoChunk;
import net.minecraft.world.level.levelgen.densityfunction.DensityFunction;
import org.junit.ClassRule;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.TemporaryFolder;
import java.io.File;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Set;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
public final class WorldgenExportDeepCaves263Test {
@ClassRule
public static final PaperRegistryIsolation registryIsolation = new PaperRegistryIsolation();
private static final long SEED = 1337L;
private static final String NAMESPACE = WorldgenExportRequest.DEFAULT_NAMESPACE;
private static final String DIMENSION_ID = NAMESPACE + ":" + ExportTestPacks.DIMENSION;
private static final int MIN_Y = -64;
private static final int LAVA_Y = -62;
private static final int LAVA_LINE = -54;
private static final int CAVE_TOP = -14;
private static final int CHUNK_WIDTH = 16;
private static final String CAVE_FIELD = "cave/field";
private static final String DIMENSION = """
{
"regions": ["region"],
"dimensionHeight": {"min": -64, "max": 192},
"fluidHeight": 63,
"caveLavaHeight": 2,
"caveProfile": {
"enabled": true,
"verticalRange": {"min": 0, "max": 50},
"verticalEdgeFade": 0,
"baseDensityStyle": {"style": "SIMPLEX", "zoom": 0.6},
"detailDensityStyle": {"style": "SIMPLEX", "zoom": 0.3},
"baseWeight": 1.0,
"detailWeight": 0.0,
"densityThreshold": {"min": 5.0, "max": 5.0, "style": {"style": "SIMPLEX"}},
"thresholdBias": 0.0,
"surfaceClearance": 0,
"allowSurfaceBreak": false,
"allowFluid": false,
"allowLava": true,
"modules": []
}
}
""";
@Rule
public TemporaryFolder temporaryFolder = new TemporaryFolder();
@Rule
public ExportTestPacks.BoundPlatform platform = new ExportTestPacks.BoundPlatform();
@Test
public void deepCavesOpenAsAirAboveTheIrisLavaBand() throws IOException {
WorldgenExportResult result = export();
assertTrue(result.message() + " " + result.diagnostics(), result.isSuccess());
JsonObject head = JsonParser.parseString(Files.readString(result.output().resolve("data/" + NAMESPACE + "/worldgen/material_rule/"
+ ExportTestPacks.DIMENSION + ".json"), StandardCharsets.UTF_8)).getAsJsonObject().getAsJsonArray("sequence").get(0).getAsJsonObject();
assertEquals("minecraft:ore_vein", head.get("type").getAsString());
String opening = head.get("density").getAsString();
assertTrue(opening, Files.isRegularFile(densityFunction(result.output(), opening)));
try (WorldgenExportSampler sampler = WorldgenExportSampler.open(result.output(), WorldgenExportMode.OVERWORLD, DIMENSION_ID, SEED)) {
ProtoChunk chunk = sampler.terrainChunk(0, 0);
for (int x = 0; x < CHUNK_WIDTH; x++) {
for (int z = 0; z < CHUNK_WIDTH; z++) {
for (int y = MIN_Y; y <= CAVE_TOP; y++) {
BlockState state = chunk.getBlockState(new BlockPos(x, y, z));
String where = state + " at " + x + "," + y + "," + z;
if (y <= LAVA_Y) {
assertTrue(where, state.is(Blocks.LAVA));
} else {
assertTrue(where, state.isAir());
}
}
}
}
assertTrue(chunk.getBlockState(new BlockPos(0, LAVA_LINE - 1, 0)).isAir());
}
}
@Test
public void deepCaveRuleSharesTheCaveFieldCacheWithTheFinalDensity() throws IOException {
WorldgenExportResult result = export();
assertTrue(result.message() + " " + result.diagnostics(), result.isSuccess());
try (WorldgenExportSampler sampler = WorldgenExportSampler.open(result.output(), WorldgenExportMode.OVERWORLD, DIMENSION_ID, SEED)) {
CompiledCaches caches = CompiledCaches.open(sampler.registries(), SEED);
DensityFunction opening = caches.oreVeins().getFirst().density();
int field = caches.rootId(caches.function(NAMESPACE + ":" + CAVE_FIELD));
Set<Integer> finalDensity = caches.ids(caches.finalDensity());
assertTrue(caches.ids(opening).toString(), caches.ids(opening).contains(field));
assertTrue(finalDensity.toString(), finalDensity.contains(field));
assertTrue(finalDensity.toString(), finalDensity.contains(caches.rootId(opening)));
}
}
private WorldgenExportResult export() throws IOException {
File pack = ExportTestPacks.withDimension(temporaryFolder.newFolder("pack"), DIMENSION);
WorldgenExportRequest request = WorldgenExportRequest.builder()
.pack(pack.toPath())
.dimension(ExportTestPacks.DIMENSION)
.exportRoot(temporaryFolder.newFolder("exports").toPath())
.format("directory")
.build();
return WorldgenDatapackExporter.create().export(request, WorldgenExportProgress.NONE);
}
private static Path densityFunction(Path datapack, String id) {
int separator = id.indexOf(':');
return datapack.resolve("data/" + id.substring(0, separator) + "/worldgen/density_function/" + id.substring(separator + 1) + ".json");
}
}
@@ -0,0 +1,75 @@
package art.arcane.iris.probe;
import art.arcane.iris.pack.datapack.export.ExportTestPacks;
import art.arcane.iris.pack.datapack.export.WorldgenDatapackExporter;
import art.arcane.iris.pack.datapack.export.WorldgenExportMode;
import art.arcane.iris.pack.datapack.export.WorldgenExportProgress;
import art.arcane.iris.pack.datapack.export.WorldgenExportRequest;
import art.arcane.iris.pack.datapack.export.WorldgenExportResult;
import org.junit.ClassRule;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.TemporaryFolder;
import java.io.File;
import java.io.IOException;
import java.nio.file.Path;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;
public final class WorldgenExportPipeline263Test {
@ClassRule
public static final PaperRegistryIsolation registryIsolation = new PaperRegistryIsolation();
private static final long SEED = 1337L;
private static final String DIMENSION_ID = WorldgenExportRequest.DEFAULT_NAMESPACE + ":" + ExportTestPacks.DIMENSION;
private static final int[] SAMPLE_Y = {-64, 0, 64, 200};
private static final int LATTICE = 8;
private static final int LATTICE_STEP = 61;
@Rule
public TemporaryFolder temporaryFolder = new TemporaryFolder();
@Rule
public ExportTestPacks.BoundPlatform platform = new ExportTestPacks.BoundPlatform();
@Test
public void exportedPackLoadsAndSamples() throws IOException {
File pack = ExportTestPacks.basic(temporaryFolder.newFolder("pack"));
Path root = temporaryFolder.newFolder("exports").toPath();
WorldgenExportRequest request = WorldgenExportRequest.builder()
.pack(pack.toPath())
.dimension(ExportTestPacks.DIMENSION)
.exportRoot(root)
.format("directory")
.build();
WorldgenExportResult result = WorldgenDatapackExporter.create().export(request, WorldgenExportProgress.NONE);
assertTrue(result.message() + " " + result.diagnostics(), result.isSuccess());
try (WorldgenExportSampler sampler = WorldgenExportSampler.open(result.output(), WorldgenExportMode.OVERWORLD, DIMENSION_ID, SEED)) {
assertFalse(sampler.densityFunctionIds().isEmpty());
for (String id : sampler.densityFunctionIds()) {
for (int y : SAMPLE_Y) {
for (int xIndex = 0; xIndex < LATTICE; xIndex++) {
for (int zIndex = 0; zIndex < LATTICE; zIndex++) {
int x = (xIndex - LATTICE / 2) * LATTICE_STEP;
int z = (zIndex - LATTICE / 2) * LATTICE_STEP;
float value = sampler.density(id, x, y, z);
assertTrue(id + " at " + x + "," + y + "," + z + " = " + value, Float.isFinite(value));
}
}
}
}
for (int xIndex = 0; xIndex < LATTICE; xIndex++) {
for (int zIndex = 0; zIndex < LATTICE; zIndex++) {
int x = (xIndex - LATTICE / 2) * LATTICE_STEP;
int z = (zIndex - LATTICE / 2) * LATTICE_STEP;
int height = sampler.baseHeight(x, z);
assertTrue("height " + height + " at " + x + "," + z, height >= sampler.minY() && height <= sampler.minY() + sampler.height());
}
}
}
}
}
@@ -0,0 +1,97 @@
package art.arcane.iris.probe;
import art.arcane.iris.pack.datapack.export.ExportTestPacks;
import art.arcane.iris.pack.datapack.export.WorldgenDatapackExporter;
import art.arcane.iris.pack.datapack.export.WorldgenExportMode;
import art.arcane.iris.pack.datapack.export.WorldgenExportProgress;
import art.arcane.iris.pack.datapack.export.WorldgenExportRequest;
import art.arcane.iris.pack.datapack.export.WorldgenExportResult;
import org.junit.ClassRule;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.TemporaryFolder;
import java.io.File;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import static org.junit.Assert.assertArrayEquals;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
public final class WorldgenExportSamplerThreadsTest {
@ClassRule
public static final PaperRegistryIsolation registryIsolation = new PaperRegistryIsolation();
private static final long SEED = 1337L;
private static final String DIMENSION_ID = WorldgenExportRequest.DEFAULT_NAMESPACE + ":" + ExportTestPacks.DIMENSION;
private static final String FINAL_DENSITY_ID = DIMENSION_ID + "/final_density";
private static final int THREADS = 8;
private static final int POINTS = 2048;
private static final int ROW = 64;
private static final int STEP = 37;
private static final int ROUNDS = 3;
@Rule
public TemporaryFolder temporaryFolder = new TemporaryFolder();
@Rule
public ExportTestPacks.BoundPlatform platform = new ExportTestPacks.BoundPlatform();
@Test
public void concurrentSamplingMatchesSequentialSampling() throws Exception {
File pack = ExportTestPacks.basic(temporaryFolder.newFolder("pack"));
WorldgenExportRequest request = WorldgenExportRequest.builder()
.pack(pack.toPath())
.dimension(ExportTestPacks.DIMENSION)
.exportRoot(temporaryFolder.newFolder("exports").toPath())
.format("directory")
.build();
WorldgenExportResult result = WorldgenDatapackExporter.create().export(request, WorldgenExportProgress.NONE);
assertTrue(result.message() + " " + result.diagnostics(), result.isSuccess());
try (WorldgenExportSampler sampler = WorldgenExportSampler.open(result.output(), WorldgenExportMode.OVERWORLD, DIMENSION_ID, SEED)) {
assertTrue(sampler.densityFunctionIds().toString(), sampler.densityFunctionIds().contains(FINAL_DENSITY_ID));
Sample expected = sweep(sampler, 0, 1).call();
ExecutorService executor = Executors.newFixedThreadPool(THREADS);
try {
List<Future<Sample>> futures = new ArrayList<>(THREADS);
for (int thread = 0; thread < THREADS; thread++) {
futures.add(executor.submit(sweep(sampler, thread * (POINTS / THREADS), ROUNDS)));
}
for (Future<Sample> future : futures) {
Sample sample = future.get();
assertArrayEquals(expected.density(), sample.density(), 0.0F);
assertEquals(List.of(expected.biome()), List.of(sample.biome()));
}
} finally {
executor.shutdownNow();
}
}
}
private static Callable<Sample> sweep(WorldgenExportSampler sampler, int offset, int rounds) {
return () -> {
float[] density = new float[POINTS];
String[] biome = new String[POINTS];
for (int round = 0; round < rounds; round++) {
for (int step = 0; step < POINTS; step++) {
int index = (offset + step) % POINTS;
int x = (index % ROW - ROW / 2) * STEP;
int y = (index * 7) % 256 - 64;
int z = (index / ROW - POINTS / ROW / 2) * STEP;
density[index] = sampler.density(FINAL_DENSITY_ID, x, y, z);
biome[index] = sampler.biome(x, y, z);
}
}
return new Sample(density, biome);
};
}
private record Sample(float[] density, String[] biome) {
}
}
@@ -0,0 +1,90 @@
package art.arcane.iris.probe;
import art.arcane.iris.pack.datapack.export.ExportTestPacks;
import art.arcane.iris.pack.datapack.export.WorldgenDatapackExporter;
import art.arcane.iris.pack.datapack.export.WorldgenExportMode;
import art.arcane.iris.pack.datapack.export.WorldgenExportProgress;
import art.arcane.iris.pack.datapack.export.WorldgenExportRequest;
import art.arcane.iris.pack.datapack.export.WorldgenExportResult;
import art.arcane.iris.testsupport.CompatTestPlatform;
import net.minecraft.core.BlockPos;
import net.minecraft.world.level.block.Blocks;
import net.minecraft.world.level.block.state.BlockState;
import net.minecraft.world.level.chunk.ProtoChunk;
import org.junit.ClassRule;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.TemporaryFolder;
import java.io.File;
import java.io.IOException;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
public final class WorldgenExportSeaLevel263Test {
@ClassRule
public static final PaperRegistryIsolation registryIsolation = new PaperRegistryIsolation();
private static final long SEED = 1337L;
private static final String DIMENSION_ID = WorldgenExportRequest.DEFAULT_NAMESPACE + ":" + ExportTestPacks.DIMENSION;
private static final int FLUID_HEIGHT = 40;
private static final int SEA_FLOOR = 28;
private static final int CHUNK_WIDTH = 16;
private static final String DIMENSION = """
{
"regions": ["region"],
"dimensionHeight": {"min": -64, "max": 192},
"fluidHeight": 40,
"landChance": 0,
"carvingEnabled": false
}
""";
@Rule
public TemporaryFolder temporaryFolder = new TemporaryFolder();
@Rule
public ExportTestPacks.BoundPlatform platform = new ExportTestPacks.BoundPlatform();
@Test
public void waterFillsThroughTheIrisFluidHeight() throws IOException {
WorldgenExportResult result = export();
assertTrue(result.message() + " " + result.diagnostics(), result.isSuccess());
try (WorldgenExportSampler sampler = WorldgenExportSampler.open(result.output(), WorldgenExportMode.OVERWORLD, DIMENSION_ID, SEED)) {
ProtoChunk chunk = sampler.terrainChunk(0, 0);
for (int x = 0; x < CHUNK_WIDTH; x++) {
for (int z = 0; z < CHUNK_WIDTH; z++) {
assertSolid(chunk.getBlockState(new BlockPos(x, SEA_FLOOR, z)), x, z);
for (int y = SEA_FLOOR + 1; y <= FLUID_HEIGHT; y++) {
BlockState state = chunk.getBlockState(new BlockPos(x, y, z));
assertTrue(state + " at " + x + "," + y + "," + z, state.is(Blocks.WATER));
}
BlockState above = chunk.getBlockState(new BlockPos(x, FLUID_HEIGHT + 1, z));
assertTrue(above + " above the fluid height at " + x + "," + z, above.isAir());
}
}
assertEquals(FLUID_HEIGHT + 1, sampler.seaLevel());
}
}
private static void assertSolid(BlockState state, int x, int z) {
assertTrue(state + " sea floor at " + x + "," + z, !state.isAir() && state.getFluidState().isEmpty());
}
private WorldgenExportResult export() throws IOException {
File pack = ExportTestPacks.withDimension(temporaryFolder.newFolder("pack"), DIMENSION);
CompatTestPlatform.write(pack, "generators/flat.json", "{\"composite\":[]}");
CompatTestPlatform.write(pack, "biomes/ocean.json", """
{"name":"Ocean","generators":[{"generator":"flat","min":-12,"max":-12}]}
""");
WorldgenExportRequest request = WorldgenExportRequest.builder()
.pack(pack.toPath())
.dimension(ExportTestPacks.DIMENSION)
.exportRoot(temporaryFolder.newFolder("exports").toPath())
.format("directory")
.build();
return WorldgenDatapackExporter.create().export(request, WorldgenExportProgress.NONE);
}
}
@@ -0,0 +1,165 @@
package art.arcane.iris.probe;
import art.arcane.iris.pack.datapack.export.ExportTestPacks;
import art.arcane.iris.pack.datapack.export.WorldgenDatapackExporter;
import art.arcane.iris.pack.datapack.export.WorldgenExportMode;
import art.arcane.iris.pack.datapack.export.WorldgenExportProgress;
import art.arcane.iris.pack.datapack.export.WorldgenExportRequest;
import art.arcane.iris.pack.datapack.export.WorldgenExportResult;
import net.minecraft.core.BlockPos;
import net.minecraft.core.registries.BuiltInRegistries;
import net.minecraft.core.registries.Registries;
import net.minecraft.nbt.CompoundTag;
import net.minecraft.nbt.NbtAccounter;
import net.minecraft.nbt.NbtIo;
import net.minecraft.resources.Identifier;
import net.minecraft.resources.ResourceKey;
import net.minecraft.world.level.block.Blocks;
import net.minecraft.world.level.block.Rotation;
import net.minecraft.world.level.levelgen.GenerationStep;
import net.minecraft.world.level.levelgen.structure.BoundingBox;
import net.minecraft.world.level.levelgen.structure.PoolElementStructurePiece;
import net.minecraft.world.level.levelgen.structure.Structure;
import net.minecraft.world.level.levelgen.structure.StructureSet;
import net.minecraft.world.level.levelgen.structure.StructureStart;
import net.minecraft.world.level.levelgen.structure.placement.RandomSpreadStructurePlacement;
import net.minecraft.world.level.levelgen.structure.pools.ListPoolElement;
import net.minecraft.world.level.levelgen.structure.structures.JigsawStructure;
import net.minecraft.world.level.levelgen.structure.templatesystem.StructurePlaceSettings;
import net.minecraft.world.level.levelgen.structure.templatesystem.StructureTemplate;
import org.junit.ClassRule;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.TemporaryFolder;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.List;
import java.util.stream.Stream;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;
public final class WorldgenExportSubterrain263Test {
@ClassRule
public static final PaperRegistryIsolation registryIsolation = new PaperRegistryIsolation();
private static final long SEED = 1337L;
private static final String DIMENSION_ID = WorldgenExportRequest.DEFAULT_NAMESPACE + ":" + ExportTestPacks.DIMENSION;
private static final String POOLS = WorldgenExportRequest.DEFAULT_NAMESPACE + ":subterrain/pools";
private static final String TUBES = WorldgenExportRequest.DEFAULT_NAMESPACE + ":subterrain/tubes";
private static final String ANCHOR = WorldgenExportRequest.DEFAULT_NAMESPACE + ":subterrain/anchor";
private static final String TEMPLATES = "data/" + WorldgenExportRequest.DEFAULT_NAMESPACE + "/structure/subterrain/";
private static final int CHUNK_X = 3;
private static final int CHUNK_Z = -2;
private static final int POOLS_CENTER_Y = -49;
private static final int POOLS_BELOW_CENTER = 10;
private static final int POOLS_EXTENT = 10;
private static final int MAX_REACH = 128;
private static final String DIMENSION = """
{"regions": ["region"], "subterrainFeatures": [
{"id": "pools", "family": "CENOTE", "worldYRange": {"min": -60, "max": -40}, "spacing": 64, "probability": 1.0,
"radius": 8, "height": 12, "fluid": "WATER", "fluidDepth": 2, "shapeWarp": 0.5, "pillarSpacing": 0,
"formationFraction": 0.0, "solid": "minecraft:stone"},
{"id": "tubes", "family": "LAVA_TUBE", "allowedRegions": ["region"], "worldYRange": {"min": -60, "max": 40},
"spacing": 320, "probability": 0.5, "length": 300, "radius": 6, "height": 8, "fluidDepth": 2, "chimneyHeight": 0,
"shapeWarp": 0.0, "pillarSpacing": 0, "formationFraction": 0.0, "solid": "minecraft:stone"}]}
""";
@Rule
public TemporaryFolder temporaryFolder = new TemporaryFolder();
@Rule
public ExportTestPacks.BoundPlatform platform = new ExportTestPacks.BoundPlatform();
@Test
public void subterrainRoomsLoadAndPlaceThroughRealCodecs() throws IOException {
File pack = ExportTestPacks.withDimension(temporaryFolder.newFolder("pack"), DIMENSION);
WorldgenExportRequest request = WorldgenExportRequest.builder()
.pack(pack.toPath())
.dimension(ExportTestPacks.DIMENSION)
.exportRoot(temporaryFolder.newFolder("exports").toPath())
.format("directory")
.build();
WorldgenExportResult result = WorldgenDatapackExporter.create().export(request, WorldgenExportProgress.NONE);
assertTrue(result.message() + " " + result.diagnostics(), result.isSuccess());
try (WorldgenExportSampler sampler = WorldgenExportSampler.open(result.output(), WorldgenExportMode.OVERWORLD, DIMENSION_ID, SEED)) {
assertRegistered(sampler, POOLS, 4, 2);
assertRegistered(sampler, TUBES, 20, 10);
int cornerX = CHUNK_X << 4;
int cornerZ = CHUNK_Z << 4;
BoundingBox pools = placedBox(sampler, POOLS);
assertEquals(cornerX - POOLS_EXTENT, pools.minX());
assertEquals(cornerX + POOLS_EXTENT, pools.maxX());
assertEquals(cornerZ - POOLS_EXTENT, pools.minZ());
assertEquals(cornerZ + POOLS_EXTENT, pools.maxZ());
assertEquals(POOLS_CENTER_Y - POOLS_BELOW_CENTER, pools.minY());
BoundingBox tubes = placedBox(sampler, TUBES);
assertEquals(2 * MAX_REACH + 1, Math.max(tubes.getXSpan(), tubes.getZSpan()));
assertTrue(tubes.toString(), tubes.minX() >= cornerX - MAX_REACH && tubes.maxX() <= cornerX + MAX_REACH);
assertTrue(tubes.toString(), tubes.minZ() >= cornerZ - MAX_REACH && tubes.maxZ() <= cornerZ + MAX_REACH);
assertTrue(tubes.toString(), tubes.minY() >= -60 && tubes.maxY() <= 40);
}
assertTemplatesResolve(result.output());
}
private static void assertRegistered(WorldgenExportSampler sampler, String id, int spacing, int separation) {
Identifier identifier = Identifier.parse(id);
Structure structure = sampler.registries().lookupOrThrow(Registries.STRUCTURE)
.getOrThrow(ResourceKey.create(Registries.STRUCTURE, identifier)).value();
assertTrue(id, structure instanceof JigsawStructure);
assertEquals(id, GenerationStep.Decoration.UNDERGROUND_STRUCTURES, structure.step());
StructureSet set = sampler.registries().lookupOrThrow(Registries.STRUCTURE_SET)
.getOrThrow(ResourceKey.create(Registries.STRUCTURE_SET, identifier)).value();
RandomSpreadStructurePlacement placement = (RandomSpreadStructurePlacement) set.placement();
assertEquals(id, spacing, placement.spacing());
assertEquals(id, separation, placement.separation());
assertTrue(id, sampler.registries().lookupOrThrow(Registries.TEMPLATE_POOL).containsKey(identifier));
}
private static BoundingBox placedBox(WorldgenExportSampler sampler, String id) throws IOException {
StructureStart start = sampler.structureStart(id, CHUNK_X, CHUNK_Z);
assertTrue(id, start.isValid());
assertEquals(id, 1, start.getPieces().size());
PoolElementStructurePiece piece = (PoolElementStructurePiece) start.getPieces().getFirst();
assertTrue(id + " " + piece.getElement(), piece.getElement() instanceof ListPoolElement);
return start.getBoundingBox();
}
private static void assertTemplatesResolve(Path datapack) throws IOException {
List<Path> templates;
try (Stream<Path> paths = Files.walk(datapack.resolve(TEMPLATES))) {
templates = paths.filter(path -> path.toString().endsWith(".nbt")).sorted().toList();
}
assertEquals(12, templates.size());
boolean lava = false;
for (Path path : templates) {
CompoundTag tag;
try (InputStream input = Files.newInputStream(path)) {
tag = NbtIo.readCompressed(input, NbtAccounter.unlimitedHeap());
}
StructureTemplate template = new StructureTemplate();
template.load(BuiltInRegistries.BLOCK, tag);
StructurePlaceSettings settings = new StructurePlaceSettings();
assertTrue(path.toString(), template.filterBlocks(BlockPos.ZERO, settings, Blocks.AIR).isEmpty());
if (path.getFileName().toString().endsWith("_core.nbt")) {
assertFalse(path.toString(), template.filterBlocks(BlockPos.ZERO, settings, Blocks.CAVE_AIR).isEmpty());
List<StructureTemplate.JigsawBlockInfo> jigsaws = template.getJigsaws(BlockPos.ZERO, Rotation.NONE);
assertEquals(path.toString(), 1, jigsaws.size());
assertEquals(ANCHOR, jigsaws.getFirst().name().toString());
lava |= !template.filterBlocks(BlockPos.ZERO, settings, Blocks.LAVA).isEmpty();
} else {
assertFalse(path.toString(), template.filterBlocks(BlockPos.ZERO, settings, Blocks.STONE).isEmpty());
}
}
assertTrue("no lava tube template holds lava", lava);
}
}
@@ -0,0 +1,111 @@
package art.arcane.iris.probe;
import art.arcane.iris.pack.datapack.DataVersion;
import art.arcane.iris.pack.datapack.export.ExportIrFixtures;
import art.arcane.iris.pack.datapack.export.WorldgenExportMode;
import art.arcane.iris.pack.datapack.export.emit.WorldgenJsonWriter;
import net.minecraft.world.level.levelgen.Heightmap;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.ClassRule;
import org.junit.Test;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Map;
import java.util.SortedMap;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
public final class WorldgenStageBenchTest {
@ClassRule
public static final PaperRegistryIsolation registryIsolation = new PaperRegistryIsolation();
private static final long SEED = 1337L;
private static final int SIDE = 2;
private static final WorldgenStageBench.CallBudget BUDGET = new WorldgenStageBench.CallBudget(32, 8, 0L);
private static HeadlessNativeRegistries loaded;
@BeforeClass
public static void loadFixture() throws IOException {
HeadlessNativeBootstrap.initialize();
SortedMap<String, byte[]> files = WorldgenJsonWriter.forTarget(DataVersion.V26_3)
.write(ExportIrFixtures.flatNoise(DataVersion.V26_3, WorldgenExportMode.PRESET));
Path directory = Files.createTempDirectory("iris-stage-bench-");
for (Map.Entry<String, byte[]> file : files.entrySet()) {
Path target = directory.resolve(file.getKey());
Files.createDirectories(target.getParent());
Files.write(target, file.getValue());
}
try (PaperRegistryScope scope = PaperRegistryScope.open()) {
loaded = HeadlessNativeRegistries.load(directory);
}
}
@AfterClass
public static void closeFixture() throws IOException {
if (loaded != null) {
loaded.close();
}
}
@Test
public void exportedFixtureRunsEveryStage() {
assertStages(bench("export", ExportIrFixtures.NAMESPACE + ":" + ExportIrFixtures.DIMENSION));
}
@Test
public void vanillaOverworldRunsEveryStage() {
assertStages(bench("vanilla", DatapackCostProbe.VANILLA_PRESET));
}
private static void assertStages(WorldgenStageBench bench) {
WorldgenStageBench.ChunkSamples samples = new WorldgenStageBench.ChunkSamples();
bench.chunks(0, 0, SIDE, samples);
bench.chunks(SIDE + 4, 0, SIDE, samples);
WorldgenStageBench.ChunkTimings timings = samples.timings();
assertEquals(2 * SIDE * SIDE, timings.chunks());
assertEquals(2 * (SIDE + 2) * (SIDE + 2), timings.biomes().wall().count());
assertEquals(2 * SIDE * SIDE, timings.noise().cpu().count());
assertEquals(2 * SIDE * SIDE, timings.surface().wall().count());
assertEquals(2 * SIDE * SIDE, timings.carverBiome().wall().count());
assertEquals(0L, timings.biomeFallbacks());
assertTrue(Double.isFinite(timings.totalMean()));
assertTrue(Double.isFinite(timings.totalCpuMean()));
assertCalls(samples(bench, Call.UNCACHED_BIOME));
assertCalls(samples(bench, Call.BASE_HEIGHT));
assertCalls(samples(bench, Call.BASE_COLUMN));
}
private static void assertCalls(WorldgenStageBench.CallTiming timing) {
assertEquals(2 * BUDGET.minimumCalls(), timing.calls());
assertEquals(2 * BUDGET.minimumCalls(), timing.wall().count());
assertTrue(Double.isFinite(timing.cpuMean()));
}
private static WorldgenStageBench.CallTiming samples(WorldgenStageBench bench, Call call) {
WorldgenStageBench.CallSamples samples = new WorldgenStageBench.CallSamples();
for (int round = 0; round < 2; round++) {
switch (call) {
case UNCACHED_BIOME -> bench.uncachedBiome(BUDGET, samples);
case BASE_HEIGHT -> bench.baseHeight(BUDGET, Heightmap.Types.WORLD_SURFACE_WG, samples);
case BASE_COLUMN -> bench.baseColumn(BUDGET, samples);
}
}
return samples.timing();
}
private static WorldgenStageBench bench(String label, String preset) {
return new WorldgenStageBench(new WorldgenStageBench.Options(label,
DatapackCostProbe.presetOverworld(loaded.registries(), preset), loaded.registries(), SEED));
}
private enum Call {
UNCACHED_BIOME,
BASE_HEIGHT,
BASE_COLUMN
}
}
@@ -0,0 +1,99 @@
package art.arcane.iris.probe.fidelity;
import art.arcane.iris.pack.datapack.DataVersion;
import art.arcane.iris.pack.datapack.export.ExportIrFixtures;
import art.arcane.iris.pack.datapack.export.WorldgenExportMode;
import art.arcane.iris.pack.datapack.export.emit.WorldgenJsonWriter;
import art.arcane.iris.pack.datapack.export.ir.AquiferDef;
import art.arcane.iris.pack.datapack.export.ir.Axis;
import art.arcane.iris.pack.datapack.export.ir.BiomeDef;
import art.arcane.iris.pack.datapack.export.ir.ClimateLayout;
import art.arcane.iris.pack.datapack.export.ir.DimensionDef;
import art.arcane.iris.pack.datapack.export.ir.ExportIr;
import art.arcane.iris.pack.datapack.export.ir.Field;
import art.arcane.iris.pack.datapack.export.ir.MaterialProgram;
import art.arcane.iris.pack.datapack.export.ir.RouterDef;
import art.arcane.iris.probe.PaperRegistryIsolation;
import art.arcane.iris.probe.WorldgenExportSampler;
import org.junit.ClassRule;
import org.junit.Test;
import java.io.IOException;
import java.util.List;
import java.util.Map;
import java.util.SortedMap;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertThrows;
import static org.junit.Assert.assertTrue;
public final class VanillaNoiseColumnSourceTest {
@ClassRule
public static final PaperRegistryIsolation registryIsolation = new PaperRegistryIsolation();
private static final long SEED = 1337L;
private static final String DIMENSION_ID = ExportIrFixtures.NAMESPACE + ":" + ExportIrFixtures.DIMENSION;
private static final String BIOME_ID = ExportIrFixtures.NAMESPACE + ":" + ExportIrFixtures.BIOME;
private static final String LOAD_KEY = "Flat Plains";
private static final int LAND_HEIGHT = 80;
private static final int SEA_FLOOR_HEIGHT = 40;
private static final int MAX_Y = ExportIrFixtures.MIN_Y + ExportIrFixtures.HEIGHT;
@Test
public void landColumnsReportTheTopSolidBlockAndStayDry() throws IOException {
try (VanillaNoiseColumnSource source = new VanillaNoiseColumnSource(open(LAND_HEIGHT), Map.of(BIOME_ID, LOAD_KEY))) {
assertEquals(ExportIrFixtures.MIN_Y, source.minY());
assertEquals(MAX_Y, source.maxY());
for (int x : new int[]{-37, 0, 113}) {
FidelityColumn column = source.column(x, -x);
assertEquals(new FidelityColumn(LOAD_KEY, LAND_HEIGHT - 1, false), column);
}
}
}
@Test
public void seaColumnsReportTheFloorUnderWater() throws IOException {
WorldgenExportSampler sampler = open(SEA_FLOOR_HEIGHT);
try (VanillaNoiseColumnSource source = new VanillaNoiseColumnSource(sampler, Map.of(BIOME_ID, LOAD_KEY))) {
FidelityStats stats = new FidelityStats(source.minY(), source.maxY());
for (int x : new int[]{-37, 0, 113}) {
FidelityColumn column = source.column(x, x);
assertEquals(new FidelityColumn(LOAD_KEY, sampler.floorHeight(x, x) - 1, true), column);
assertTrue(column.terrainY() + " under sea level", column.terrainY() < ExportIrFixtures.SEA_LEVEL - 1);
assertEquals(ExportIrFixtures.SEA_LEVEL, sampler.baseHeight(x, x));
stats.accept(column);
}
assertEquals(0.0D, stats.landFraction(), 0.0D);
}
}
@Test
public void exportedBiomeWithoutALoadKeyIsRejected() throws IOException {
try (VanillaNoiseColumnSource source = new VanillaNoiseColumnSource(open(LAND_HEIGHT), Map.of())) {
IllegalStateException failure = assertThrows(IllegalStateException.class, () -> source.column(0, 0));
assertTrue(failure.getMessage(), failure.getMessage().contains(BIOME_ID));
}
}
private static WorldgenExportSampler open(int height) throws IOException {
return WorldgenExportSampler.open(flatExport(height), WorldgenExportMode.PRESET, DIMENSION_ID, SEED);
}
private static SortedMap<String, byte[]> flatExport(int height) {
Field zero = new Field.Const(0.0);
Field level = new Field.Const(height);
Field y = new Field.Coord(Axis.Y, Field.Tiling.CLAMP, -4064.0, 4062.0, -4064.0, 4062.0);
Field finalDensity = new Field.Grid(new Field.Binary(Field.BinaryOp.MUL, new Field.Const(0.125),
new Field.Binary(Field.BinaryOp.SUB, level, y)), 4, 8);
RouterDef router = new RouterDef(zero, zero, zero, zero, zero, zero, new Field.Grid(level, 16, 1), finalDensity, AquiferDef.dry(level));
DimensionDef dimension = new DimensionDef(ExportIrFixtures.DIMENSION, ExportIrFixtures.MIN_Y, ExportIrFixtures.HEIGHT,
ExportIrFixtures.HEIGHT, ExportIrFixtures.FLUID_HEIGHT, "minecraft:stone", "minecraft:water", ExportIrFixtures.dimensionType());
ClimateLayout climate = new ClimateLayout(List.of(new ClimateLayout.ClimateEntry(BIOME_ID, ClimateLayout.ClimateAxis.TEMPERATURE, -2.0, 2.0)));
ExportIr ir = new ExportIr(ExportIrFixtures.NAMESPACE, DataVersion.V26_3, WorldgenExportMode.PRESET, dimension, List.of(), router, climate,
List.of(new BiomeDef(ExportIrFixtures.BIOME, ExportIrFixtures.biome())), MaterialProgram.rock("minecraft:stone"),
List.of(), List.of(), List.of());
return WorldgenJsonWriter.forTarget(DataVersion.V26_3).write(ir);
}
}
@@ -0,0 +1,189 @@
package art.arcane.iris.probe.fidelity;
import art.arcane.iris.pack.datapack.DataVersion;
import art.arcane.iris.pack.datapack.export.ExportIrFixtures;
import art.arcane.iris.pack.datapack.export.WorldgenExportMode;
import art.arcane.iris.pack.datapack.export.emit.WorldgenJsonWriter;
import art.arcane.iris.pack.datapack.export.ir.AquiferDef;
import art.arcane.iris.pack.datapack.export.ir.Axis;
import art.arcane.iris.pack.datapack.export.ir.BiomeDef;
import art.arcane.iris.pack.datapack.export.ir.ClimateLayout;
import art.arcane.iris.pack.datapack.export.ir.DimensionDef;
import art.arcane.iris.pack.datapack.export.ir.ExportIr;
import art.arcane.iris.pack.datapack.export.ir.Field;
import art.arcane.iris.pack.datapack.export.ir.JsonResource;
import art.arcane.iris.pack.datapack.export.ir.MaterialProgram;
import art.arcane.iris.pack.datapack.export.ir.RouterDef;
import art.arcane.iris.probe.PaperRegistryIsolation;
import art.arcane.iris.probe.WorldgenExportSampler;
import com.google.gson.JsonParser;
import org.junit.ClassRule;
import org.junit.Test;
import java.io.IOException;
import java.util.List;
import java.util.Map;
import java.util.SortedMap;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertNull;
import static org.junit.Assert.assertThrows;
import static org.junit.Assert.assertTrue;
public final class VanillaNoiseFidelitySourceTest {
@ClassRule
public static final PaperRegistryIsolation registryIsolation = new PaperRegistryIsolation();
private static final long SEED = 1337L;
private static final String DIMENSION_ID = ExportIrFixtures.NAMESPACE + ":" + ExportIrFixtures.DIMENSION;
private static final String BIOME_ID = ExportIrFixtures.NAMESPACE + ":" + ExportIrFixtures.BIOME;
private static final String LOAD_KEY = "Flat Plains";
private static final int LAND_HEIGHT = 80;
private static final int MAX_Y = ExportIrFixtures.MIN_Y + ExportIrFixtures.HEIGHT;
private static final String WALL_SHELL = ExportIrFixtures.NAMESPACE + ":cave/wall_shell";
private static final int WALL_MIN_Y = 10;
private static final int WALL_MAX_Y = 20;
private static final int CARVED_HEIGHT = 120;
private static final int VOID_MIN_Y = 96;
private static final int CAVE_MIN_Y = 64;
private static final Field BLOCK_Y = new Field.Coord(Axis.Y, Field.Tiling.CLAMP, -4064.0, 4062.0, -4064.0, 4062.0);
@Test
public void flatExportReportsTheConstantSurface() throws IOException {
WorldgenExportSampler sampler = open(LAND_HEIGHT);
try (VanillaNoiseFidelitySource source = new VanillaNoiseFidelitySource(sampler, Map.of(BIOME_ID, LOAD_KEY), WALL_SHELL)) {
assertEquals(ExportIrFixtures.MIN_Y, source.minY());
assertEquals(MAX_Y, source.maxY());
for (int x : new int[]{-37, 0, 113}) {
assertEquals(LAND_HEIGHT, sampler.baseHeight(x, -x));
}
for (int[] chunkPosition : new int[][]{{0, 0}, {-3, 5}}) {
FidelityChunk chunk = source.sample(chunkPosition[0], chunkPosition[1]);
for (int x = 0; x < 16; x++) {
for (int z = 0; z < 16; z++) {
assertEquals(LAND_HEIGHT - 1, chunk.surfaceY(x, z));
assertEquals(LOAD_KEY, chunk.surfaceBiome(x, z));
assertTrue(chunk.open(x, LAND_HEIGHT, z));
assertNull(chunk.block(x, LAND_HEIGHT - 1, z));
}
}
for (int y = ExportIrFixtures.MIN_Y; y < LAND_HEIGHT; y++) {
assertFalse("y " + y, chunk.open(7, y, 9));
assertFalse("y " + y, chunk.fluid(7, y, 9));
}
}
FidelityStats stats = FidelityScanner.scan(source, 0, 0, 1, 4, 2);
assertEquals(144L, stats.heightColumns());
assertEquals(LAND_HEIGHT - 1, stats.heightQuantile(0.1D), 0.0D);
assertEquals(LAND_HEIGHT - 1, stats.heightQuantile(0.9D), 0.0D);
assertEquals(1.0D, stats.landFraction(), 0.0D);
assertEquals(Map.of(LOAD_KEY, 144L), stats.biomeColumns());
assertEquals(0L, stats.blockColumns());
assertEquals(0L, stats.wallSolidVoxels());
}
}
@Test
public void wallShellDensityMarksWallVoxels() throws IOException {
WorldgenExportSampler sampler = WorldgenExportSampler.open(flatExport(LAND_HEIGHT, wallShell()), WorldgenExportMode.PRESET, DIMENSION_ID, SEED);
try (VanillaNoiseFidelitySource source = new VanillaNoiseFidelitySource(sampler, Map.of(BIOME_ID, LOAD_KEY), WALL_SHELL)) {
FidelityChunk chunk = source.sample(2, -1);
assertTrue(chunk.wallsKnown());
assertTrue(chunk.wall(7, WALL_MIN_Y, 9));
assertTrue(chunk.wall(7, WALL_MAX_Y - 1, 9));
assertFalse(chunk.wall(7, WALL_MIN_Y - 1, 9));
assertFalse(chunk.wall(7, WALL_MAX_Y, 9));
FidelityStats stats = FidelityScanner.scan(source, 0, 0, 1, 4, 2);
long interiorColumns = 9L * 9L;
assertEquals(interiorColumns * (LAND_HEIGHT - 1 - ExportIrFixtures.MIN_Y), stats.wallSolidVoxels());
assertEquals(interiorColumns * (WALL_MAX_Y - WALL_MIN_Y), stats.wallVoxels());
}
}
@Test
public void openSpaceOutsideTheTerrainOperandIsATerrainVoid() throws IOException {
WorldgenExportSampler sampler = WorldgenExportSampler.open(carvedExport(), WorldgenExportMode.PRESET, DIMENSION_ID, SEED);
try (VanillaNoiseFidelitySource source = new VanillaNoiseFidelitySource(sampler, Map.of(BIOME_ID, LOAD_KEY), WALL_SHELL)) {
FidelityChunk chunk = source.sample(1, -2);
assertEquals(CARVED_HEIGHT - 1, chunk.surfaceY(7, 9));
assertTrue(chunk.open(7, VOID_MIN_Y, 9));
assertFalse(chunk.terrainMass(7, VOID_MIN_Y, 9));
assertTrue(chunk.open(7, CAVE_MIN_Y + 3, 9));
assertTrue(chunk.terrainMass(7, CAVE_MIN_Y + 3, 9));
assertFalse(chunk.open(7, VOID_MIN_Y - 4, 9));
assertTrue(chunk.terrainMass(7, VOID_MIN_Y - 4, 9));
FidelityStats stats = FidelityScanner.scan(source, 0, 0, 1, 4, 2);
int voidBand = (VOID_MIN_Y - ExportIrFixtures.MIN_Y) / 16;
int caveBand = (CAVE_MIN_Y - ExportIrFixtures.MIN_Y) / 16;
assertEquals(1.0D, stats.voidShare(voidBand), 0.0D);
assertEquals(0.0D, stats.caveShare(voidBand), 0.0D);
assertEquals(1.0D, stats.caveShare(caveBand), 0.0D);
assertEquals(0.0D, stats.voidShare(caveBand), 0.0D);
}
}
@Test
public void exportWithoutAWallShellReportsNoWalls() throws IOException {
try (VanillaNoiseFidelitySource source = new VanillaNoiseFidelitySource(open(LAND_HEIGHT), Map.of(BIOME_ID, LOAD_KEY), WALL_SHELL)) {
assertFalse(source.sample(0, 0).wallsKnown());
}
}
@Test
public void exportedBiomeWithoutALoadKeyIsRejected() throws IOException {
try (VanillaNoiseFidelitySource source = new VanillaNoiseFidelitySource(open(LAND_HEIGHT), Map.of(), WALL_SHELL)) {
FidelityChunk chunk = source.sample(0, 0);
IllegalStateException failure = assertThrows(IllegalStateException.class, () -> chunk.surfaceBiome(0, 0));
assertTrue(failure.getMessage(), failure.getMessage().contains(BIOME_ID));
}
}
private static WorldgenExportSampler open(int height) throws IOException {
return WorldgenExportSampler.open(flatExport(height), WorldgenExportMode.PRESET, DIMENSION_ID, SEED);
}
private static JsonResource wallShell() {
return new JsonResource("data/" + ExportIrFixtures.NAMESPACE + "/worldgen/density_function/cave/wall_shell.json", JsonParser.parseString(
"{\"type\":\"minecraft:range_choice\",\"input\":{\"type\":\"minecraft:gradient\",\"axis\":\"y\",\"tiling\":\"clamp_to_edge\","
+ "\"from_coordinate\":-4064,\"to_coordinate\":4062,\"from_value\":-4064.0,\"to_value\":4062.0},"
+ "\"min_inclusive\":" + WALL_MIN_Y + ".0,\"max_exclusive\":" + WALL_MAX_Y + ".0,\"when_in_range\":1.0,\"when_out_of_range\":0.0}"));
}
private static SortedMap<String, byte[]> flatExport(int height, JsonResource... resources) {
Field level = new Field.Const(height);
Field finalDensity = new Field.Grid(new Field.Binary(Field.BinaryOp.MUL, new Field.Const(0.125),
new Field.Binary(Field.BinaryOp.SUB, level, BLOCK_Y)), 4, 8);
return export(height, finalDensity, resources);
}
private static SortedMap<String, byte[]> carvedExport() {
Field ramp = new Field.Binary(Field.BinaryOp.MUL, new Field.Const(0.125), new Field.Binary(Field.BinaryOp.SUB, new Field.Const(CARVED_HEIGHT), BLOCK_Y));
Field terrain = new Field.RangeIf(BLOCK_Y, VOID_MIN_Y, VOID_MIN_Y + 16, new Field.Const(-1.0), ramp);
Field caves = new Field.RangeIf(BLOCK_Y, CAVE_MIN_Y, CAVE_MIN_Y + 16, new Field.Const(-1.0), new Field.Const(1.0));
return export(CARVED_HEIGHT, new Field.Binary(Field.BinaryOp.MIN, terrain, caves));
}
private static SortedMap<String, byte[]> export(int height, Field finalDensity, JsonResource... resources) {
Field zero = new Field.Const(0.0);
Field level = new Field.Const(height);
RouterDef router = new RouterDef(zero, zero, zero, zero, zero, zero, new Field.Grid(level, 16, 1), finalDensity, AquiferDef.dry(level));
DimensionDef dimension = new DimensionDef(ExportIrFixtures.DIMENSION, ExportIrFixtures.MIN_Y, ExportIrFixtures.HEIGHT,
ExportIrFixtures.HEIGHT, ExportIrFixtures.FLUID_HEIGHT, "minecraft:stone", "minecraft:water", ExportIrFixtures.dimensionType());
ClimateLayout climate = new ClimateLayout(List.of(new ClimateLayout.ClimateEntry(BIOME_ID, ClimateLayout.ClimateAxis.TEMPERATURE, -2.0, 2.0)));
ExportIr ir = new ExportIr(ExportIrFixtures.NAMESPACE, DataVersion.V26_3, WorldgenExportMode.PRESET, dimension, List.of(), router, climate,
List.of(new BiomeDef(ExportIrFixtures.BIOME, ExportIrFixtures.biome())), MaterialProgram.rock("minecraft:stone"),
List.of(), List.of(resources), List.of());
return WorldgenJsonWriter.forTarget(DataVersion.V26_3).write(ir);
}
}
@@ -18,21 +18,41 @@
package art.arcane.iris.command; package art.arcane.iris.command;
import art.arcane.iris.pack.datapack.DatapackIngestService; import art.arcane.iris.Iris;
import art.arcane.volmlib.util.collection.KList; import art.arcane.iris.command.handlers.DimensionHandler;
import art.arcane.iris.generation.terrain.IrisDimension;
import art.arcane.iris.localization.BukkitCommandMessages;
import art.arcane.iris.localization.C; import art.arcane.iris.localization.C;
import art.arcane.iris.localization.DirectorCommandMessages;
import art.arcane.iris.localization.IrisLanguage;
import art.arcane.iris.pack.datapack.DataVersion;
import art.arcane.iris.pack.datapack.DatapackIngestService;
import art.arcane.iris.pack.datapack.export.DatapackExportDestinations;
import art.arcane.iris.pack.datapack.export.WorldgenExportMode;
import art.arcane.iris.pack.datapack.export.WorldgenExportProgress;
import art.arcane.iris.pack.datapack.export.WorldgenExportRequest;
import art.arcane.iris.pack.datapack.export.WorldgenExportResult;
import art.arcane.iris.pack.datapack.export.WorldgenExportService;
import art.arcane.iris.pack.datapack.export.WorldgenExportServices;
import art.arcane.iris.pack.datapack.export.WorldgenExportTargets;
import art.arcane.iris.pack.datapack.export.ir.ExportDiagnostic;
import art.arcane.iris.platform.bukkit.plugin.VolmitSender; import art.arcane.iris.platform.bukkit.plugin.VolmitSender;
import art.arcane.iris.spi.IrisPlatforms;
import art.arcane.iris.world.task.J; import art.arcane.iris.world.task.J;
import art.arcane.volmlib.util.collection.KList;
import art.arcane.volmlib.util.director.annotations.Director; import art.arcane.volmlib.util.director.annotations.Director;
import art.arcane.volmlib.util.director.annotations.Param; import art.arcane.volmlib.util.director.annotations.Param;
import art.arcane.volmlib.util.localization.LanguageAudience;
import art.arcane.volmlib.util.localization.MessageArgument;
import art.arcane.volmlib.util.localization.TextKey;
import java.util.List; import java.util.List;
import java.util.UUID;
import art.arcane.iris.localization.IrisLanguage;
import art.arcane.iris.localization.BukkitCommandMessages;
import art.arcane.volmlib.util.localization.MessageArgument;
@Director(name = "datapack", aliases = {"datapacks", "dp"}, description = "Download & manage external datapack imports", descriptionKey = "iris.director.commanddatapack.director.download_manage_external_datapack_imports_modrinth") @Director(name = "datapack", aliases = {"datapacks", "dp"}, description = "Download & manage external datapack imports", descriptionKey = "iris.director.commanddatapack.director.download_manage_external_datapack_imports_modrinth")
public class CommandDatapack implements DirectorExecutor { public class CommandDatapack implements DirectorExecutor {
private static final int RESOURCE_REPORTS = 4;
@Director(description = "Download/update every datapack listed in a pack dimension's 'datapackImports' and install it into the world so its structures register like vanilla", descriptionKey = "iris.director.commanddatapack.director.download_update_every_datapack_listed_pack_dimension_s_datapackimports_install_it_into", aliases = {"pull"}) @Director(description = "Download/update every datapack listed in a pack dimension's 'datapackImports' and install it into the world so its structures register like vanilla", descriptionKey = "iris.director.commanddatapack.director.download_update_every_datapack_listed_pack_dimension_s_datapackimports_install_it_into", aliases = {"pull"})
public void ingest( public void ingest(
@Param(description = "Restart the server when new datapacks are installed (required for new structures to register and generate)", descriptionKey = "iris.director.commanddatapack.param.restart_server_when_new_datapacks_are_installed_required_new_structures_register_generate", defaultValue = "false") @Param(description = "Restart the server when new datapacks are installed (required for new structures to register and generate)", descriptionKey = "iris.director.commanddatapack.param.restart_server_when_new_datapacks_are_installed_required_new_structures_register_generate", defaultValue = "false")
@@ -71,4 +91,128 @@ public class CommandDatapack implements DirectorExecutor {
) { ) {
DatapackIngestService.remove(sender(), id); DatapackIngestService.remove(sender(), id);
} }
@Director(description = "Export a dimension as a vanilla worldgen datapack", descriptionKey = "iris.director.commanddatapack.director.export_dimension_vanilla_worldgen_datapack")
public void export(
@Param(name = "dimension", description = "The dimension to export", descriptionKey = "iris.director.commanddatapack.param.export_dimension", customHandler = DimensionHandler.class)
IrisDimension dimension,
@Param(name = "version", description = "Target Minecraft version, 26.3 or later (blank uses the latest)", descriptionKey = "iris.director.commanddatapack.param.export_version", defaultValue = "")
String version,
@Param(name = "mode", description = "Output mode: preset, dimension, or overworld", descriptionKey = "iris.director.commanddatapack.param.export_mode", defaultValue = WorldgenExportRequest.DEFAULT_MODE)
String mode,
@Param(name = "namespace", description = "Namespace for the exported resources", descriptionKey = "iris.director.commanddatapack.param.export_namespace", defaultValue = WorldgenExportRequest.DEFAULT_NAMESPACE)
String namespace,
@Param(name = "output", description = "Output name in the datapack exports folder (blank uses the dimension and version)", descriptionKey = "iris.director.commanddatapack.param.export_output", defaultValue = "")
String output,
@Param(name = "format", description = "zip or directory", descriptionKey = "iris.director.commanddatapack.param.export_format", defaultValue = WorldgenExportRequest.DEFAULT_FORMAT)
String format,
@Param(name = "objects", description = "Include objects as structure templates", descriptionKey = "iris.director.commanddatapack.param.export_objects", defaultValue = "true")
boolean objects,
@Param(name = "replace", description = "Replace an existing export with the same name", descriptionKey = "iris.director.commanddatapack.param.export_replace", defaultValue = "false")
boolean replace
) {
VolmitSender sender = sender();
WorldgenExportRequest request = WorldgenExportRequest.builder()
.pack(dimension.getLoader().getDataFolder().toPath())
.dimension(dimension.getLoadKey())
.version(version)
.mode(mode)
.namespace(namespace)
.output(output)
.format(format)
.objects(objects)
.replace(replace)
.exportRoot(IrisPlatforms.get().dataFolder("exports", "datapacks").toPath())
.build();
DataVersion target;
try {
target = validateExport(request);
} catch (IllegalArgumentException invalid) {
sender.sendMessage(C.RED + IrisLanguage.text(DirectorCommandMessages.COMMAND_DATAPACK_EXPORT_INVALID, MessageArgument.untrusted("reason", invalid.getMessage())));
return;
}
WorldgenExportService service = WorldgenExportServices.get();
sender.sendMessage(C.GRAY + IrisLanguage.text(DirectorCommandMessages.COMMAND_DATAPACK_EXPORT_STARTED, MessageArgument.untrusted("dimension", request.dimension()), MessageArgument.untrusted("version", target.getVersion())));
UUID audience = LanguageAudience.current();
ExportFeedback feedback = new ExportFeedback(sender);
J.a(() -> LanguageAudience.run(audience, () -> runExport(service, request, feedback)));
}
static DataVersion validateExport(WorldgenExportRequest request) {
DataVersion target = WorldgenExportTargets.resolve(request.version());
WorldgenExportMode.parse(request.mode());
DatapackExportDestinations.parseFormat(request.format());
return target;
}
private static void runExport(WorldgenExportService service, WorldgenExportRequest request, ExportFeedback feedback) {
WorldgenExportResult result;
try {
result = service.export(request, feedback);
} catch (Throwable failure) {
Iris.reportError("Datapack export failed for '" + request.dimension() + "'.", failure);
feedback.send(C.RED, IrisLanguage.text(DirectorCommandMessages.COMMAND_DATAPACK_EXPORT_FAILED, MessageArgument.untrusted("reason", IrisLanguage.errorDetail(failure))));
return;
}
reportExport(feedback, result);
}
private static void reportExport(ExportFeedback feedback, WorldgenExportResult result) {
for (ExportDiagnostic diagnostic : result.diagnostics()) {
feedback.send(diagnosticColor(diagnostic.severity()), IrisLanguage.text(DirectorCommandMessages.COMMAND_DATAPACK_EXPORT_DIAGNOSTIC, MessageArgument.untrusted("key", diagnosticLabel(diagnostic)), MessageArgument.untrusted("message", diagnostic.message())));
}
if (result.isSuccess()) {
feedback.send(C.GREEN, IrisLanguage.text(DirectorCommandMessages.COMMAND_DATAPACK_EXPORT_EXPORTED, MessageArgument.untrusted("path", result.output())));
return;
}
TextKey outcome = result.status() == WorldgenExportResult.Status.REJECTED
? DirectorCommandMessages.COMMAND_DATAPACK_EXPORT_REJECTED
: DirectorCommandMessages.COMMAND_DATAPACK_EXPORT_FAILED;
feedback.send(C.RED, IrisLanguage.text(outcome, MessageArgument.untrusted("reason", result.message())));
}
private static C diagnosticColor(ExportDiagnostic.Severity severity) {
return switch (severity) {
case ERROR -> C.RED;
case WARNING -> C.YELLOW;
case INFO -> C.GRAY;
};
}
private static String diagnosticLabel(ExportDiagnostic diagnostic) {
return diagnostic.resource().isEmpty()
? diagnostic.limit().name()
: diagnostic.limit().name() + " " + diagnostic.resource();
}
private static final class ExportFeedback implements WorldgenExportProgress {
private final VolmitSender sender;
private ExportFeedback(VolmitSender sender) {
this.sender = sender;
}
@Override
public void stage(String stage) {
send(C.GRAY, IrisLanguage.text(DirectorCommandMessages.COMMAND_DATAPACK_EXPORT_STAGE, MessageArgument.untrusted("stage", stage)));
}
@Override
public void resources(int done, int total) {
int stride = Math.max(1, total / RESOURCE_REPORTS);
if (done != total && done % stride != 0) {
return;
}
send(C.GRAY, IrisLanguage.text(DirectorCommandMessages.COMMAND_DATAPACK_EXPORT_RESOURCES, MessageArgument.untrusted("done", done), MessageArgument.untrusted("total", total)));
}
private void send(C color, String message) {
String line = color + message;
if (sender.isPlayer()) {
J.runEntity(sender.player(), () -> sender.sendMessage(line));
return;
}
sender.sendMessage(line);
}
}
} }
@@ -21,7 +21,8 @@ import art.arcane.iris.structure.StructureImporter;
import art.arcane.iris.structure.VillageImporter; import art.arcane.iris.structure.VillageImporter;
import art.arcane.iris.structure.authoring.StructureKey; import art.arcane.iris.structure.authoring.StructureKey;
import art.arcane.iris.structure.authoring.StructureWriteResult; import art.arcane.iris.structure.authoring.StructureWriteResult;
import art.arcane.iris.structure.export.VanillaJigsawExportFormat; import art.arcane.iris.pack.datapack.export.DatapackExportDestinations;
import art.arcane.iris.pack.datapack.export.DatapackExportFormat;
import art.arcane.iris.structure.export.VanillaJigsawExportRequest; import art.arcane.iris.structure.export.VanillaJigsawExportRequest;
import art.arcane.iris.structure.export.VanillaJigsawExportSettings; import art.arcane.iris.structure.export.VanillaJigsawExportSettings;
import art.arcane.iris.structure.export.VanillaJigsawExportSource; import art.arcane.iris.structure.export.VanillaJigsawExportSource;
@@ -420,9 +421,9 @@ public class CommandJigsaw implements DirectorExecutor {
sendError("Wait for the pending autosave or discard the edits before exporting the on-disk graph."); sendError("Wait for the pending autosave or discard the edits before exporting the on-disk graph.");
return; return;
} }
VanillaJigsawExportFormat exportFormat; DatapackExportFormat exportFormat;
try { try {
exportFormat = CommandJigsawExports.parseFormat(format); exportFormat = DatapackExportDestinations.parseFormat(format);
} catch (IllegalArgumentException exception) { } catch (IllegalArgumentException exception) {
sendError(exception.getMessage()); sendError(exception.getMessage());
return; return;
@@ -431,7 +432,7 @@ public class CommandJigsaw implements DirectorExecutor {
.toPath().toAbsolutePath().normalize(); .toPath().toAbsolutePath().normalize();
Path destination; Path destination;
try { try {
destination = CommandJigsawExports.resolveExportDestination(exportRoot, output, exportFormat); destination = DatapackExportDestinations.resolve(exportRoot, output, exportFormat);
} catch (IllegalArgumentException exception) { } catch (IllegalArgumentException exception) {
sendError(exception.getMessage()); sendError(exception.getMessage());
return; return;
@@ -4,7 +4,7 @@ import art.arcane.iris.Iris;
import art.arcane.iris.studio.jigsaw.JigsawStudioService; import art.arcane.iris.studio.jigsaw.JigsawStudioService;
import art.arcane.iris.structure.export.VanillaJigsawDatapackExporter; import art.arcane.iris.structure.export.VanillaJigsawDatapackExporter;
import art.arcane.iris.structure.export.VanillaJigsawExportDiagnostic; import art.arcane.iris.structure.export.VanillaJigsawExportDiagnostic;
import art.arcane.iris.structure.export.VanillaJigsawExportFormat; import art.arcane.iris.pack.datapack.export.DatapackExportDestinations;
import art.arcane.iris.structure.export.VanillaJigsawExportRequest; import art.arcane.iris.structure.export.VanillaJigsawExportRequest;
import art.arcane.iris.structure.export.VanillaJigsawExportResult; import art.arcane.iris.structure.export.VanillaJigsawExportResult;
import art.arcane.iris.localization.C; import art.arcane.iris.localization.C;
@@ -13,18 +13,14 @@ import art.arcane.iris.world.task.J;
import org.bukkit.entity.Player; import org.bukkit.entity.Player;
import java.nio.file.Path; import java.nio.file.Path;
import java.util.Locale;
import java.util.Objects; import java.util.Objects;
import java.util.Set; import java.util.Set;
import java.util.UUID; import java.util.UUID;
import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicBoolean; import java.util.concurrent.atomic.AtomicBoolean;
import java.util.regex.Pattern;
final class CommandJigsawExports { final class CommandJigsawExports {
private static final Pattern EXPORT_OUTPUT_NAME = Pattern.compile("[a-zA-Z0-9_-][a-zA-Z0-9._-]{0,127}");
private static final Set<UUID> EXPORTING_PLAYERS = ConcurrentHashMap.newKeySet(); private static final Set<UUID> EXPORTING_PLAYERS = ConcurrentHashMap.newKeySet();
private static final Set<Path> EXPORT_DESTINATIONS = ConcurrentHashMap.newKeySet();
private CommandJigsawExports() { private CommandJigsawExports() {
} }
@@ -45,11 +41,11 @@ final class CommandJigsawExports {
static boolean beginExport(UUID playerId, Path destination) { static boolean beginExport(UUID playerId, Path destination) {
Objects.requireNonNull(playerId); Objects.requireNonNull(playerId);
Path normalizedDestination = Objects.requireNonNull(destination).toAbsolutePath().normalize(); Objects.requireNonNull(destination);
if (!EXPORTING_PLAYERS.add(playerId)) { if (!EXPORTING_PLAYERS.add(playerId)) {
return false; return false;
} }
if (EXPORT_DESTINATIONS.add(normalizedDestination)) { if (DatapackExportDestinations.claim(destination)) {
return true; return true;
} }
EXPORTING_PLAYERS.remove(playerId); EXPORTING_PLAYERS.remove(playerId);
@@ -58,7 +54,7 @@ final class CommandJigsawExports {
static void finishExport(UUID playerId, Path destination) { static void finishExport(UUID playerId, Path destination) {
EXPORTING_PLAYERS.remove(Objects.requireNonNull(playerId)); EXPORTING_PLAYERS.remove(Objects.requireNonNull(playerId));
EXPORT_DESTINATIONS.remove(Objects.requireNonNull(destination).toAbsolutePath().normalize()); DatapackExportDestinations.release(destination);
} }
static void runExport(ExportOperation operation) { static void runExport(ExportOperation operation) {
@@ -102,35 +98,6 @@ final class CommandJigsawExports {
+ result.status() + ": " + result.output()); + result.status() + ": " + result.output());
} }
static VanillaJigsawExportFormat parseFormat(String value) {
return switch (value.trim().toLowerCase(Locale.ROOT)) {
case "zip" -> VanillaJigsawExportFormat.ZIP;
case "directory", "dir", "folder" -> VanillaJigsawExportFormat.DIRECTORY;
default -> throw new IllegalArgumentException("Export format must be directory or zip.");
};
}
static Path resolveExportDestination(
Path exportRoot,
String output,
VanillaJigsawExportFormat format
) {
Path root = Objects.requireNonNull(exportRoot, "Jigsaw export root").toAbsolutePath().normalize();
String requested = Objects.requireNonNull(output, "Jigsaw export output name");
String name = requested.trim();
if (!name.equals(requested) || !EXPORT_OUTPUT_NAME.matcher(name).matches()) {
throw new IllegalArgumentException(
"Export output must be one file name using letters, numbers, '.', '_', or '-'.");
}
String fileName = format == VanillaJigsawExportFormat.ZIP && !name.endsWith(".zip")
? name + ".zip" : name;
Path destination = root.resolve(fileName).normalize();
if (destination.equals(root) || !root.equals(destination.getParent())) {
throw new IllegalArgumentException("Export output must be one artifact inside the Studio exports folder.");
}
return destination;
}
record ExportOperation( record ExportOperation(
Player player, Player player,
VolmitSender sender, VolmitSender sender,
@@ -0,0 +1,104 @@
package art.arcane.iris.command;
import art.arcane.iris.command.handlers.DimensionHandler;
import art.arcane.iris.generation.terrain.IrisDimension;
import art.arcane.iris.localization.IrisLanguage;
import art.arcane.iris.pack.datapack.DataVersion;
import art.arcane.iris.pack.datapack.export.WorldgenExportRequest;
import art.arcane.volmlib.util.director.annotations.Director;
import art.arcane.volmlib.util.director.annotations.Param;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.TemporaryFolder;
import java.lang.reflect.Method;
import java.lang.reflect.Parameter;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertSame;
import static org.junit.Assert.assertThrows;
public class CommandDatapackExportContractTest {
private static final List<String> NAMES = List.of(
"dimension", "version", "mode", "namespace", "output", "format", "objects", "replace");
private static final List<String> DEFAULTS = List.of(
Param.NO_DEFAULT, "", "overworld", "iris_export", "", "zip", "true", "false");
@Rule
public TemporaryFolder temporaryFolder = new TemporaryFolder();
@Test
public void exportParametersAreNamedInOrder() throws Exception {
List<String> names = new ArrayList<>();
for (Parameter parameter : exportMethod().getParameters()) {
names.add(parameter.getAnnotation(Param.class).name());
}
assertEquals(NAMES, names);
}
@Test
public void dimensionIsPositionalAndEveryOtherParameterIsKeyedWithItsDefault() throws Exception {
Parameter[] parameters = exportMethod().getParameters();
List<String> defaults = new ArrayList<>();
for (Parameter parameter : parameters) {
defaults.add(parameter.getAnnotation(Param.class).defaultValue());
}
assertEquals(DEFAULTS, defaults);
assertEquals(DimensionHandler.class, parameters[0].getAnnotation(Param.class).customHandler());
}
@Test
public void exportCommandAndParametersUseRegisteredDescriptions() throws Exception {
Method export = exportMethod();
Director director = export.getAnnotation(Director.class);
assertNotNull(director);
assertEquals(0L, Arrays.stream(director.aliases()).filter((String alias) -> !alias.isBlank()).count());
assertNotNull(director.descriptionKey(), IrisLanguage.catalog().key(director.descriptionKey()));
for (Parameter parameter : export.getParameters()) {
String key = parameter.getAnnotation(Param.class).descriptionKey();
assertNotNull(key, IrisLanguage.catalog().key(key));
}
}
@Test
public void validRequestResolvesItsTarget() {
assertSame(DataVersion.V26_3, CommandDatapack.validateExport(request().build()));
assertSame(DataVersion.V26_3, CommandDatapack.validateExport(request().version("26.3").build()));
}
@Test
public void unsupportedArgumentsAreRejectedWithSharedMessages() {
assertEquals("Unsupported datapack export version '1.21.1'. Supported: 26.3",
reason(request().version("1.21.1").build()));
assertEquals("Unsupported datapack export version '26.2'. Supported: 26.3",
reason(request().version("26.2").build()));
assertEquals("Unknown export mode 'nether'. Expected: preset, dimension, overworld",
reason(request().mode("nether").build()));
assertEquals("Export format must be directory or zip.", reason(request().format("rar").build()));
}
private WorldgenExportRequest.Builder request() {
Path root = temporaryFolder.getRoot().toPath();
return WorldgenExportRequest.builder()
.pack(root.resolve("packs").resolve("overworld"))
.dimension("overworld")
.exportRoot(root.resolve("exports"));
}
private static String reason(WorldgenExportRequest request) {
return assertThrows(IllegalArgumentException.class, () -> CommandDatapack.validateExport(request)).getMessage();
}
private static Method exportMethod() throws NoSuchMethodException {
return CommandDatapack.class.getDeclaredMethod("export", IrisDimension.class, String.class, String.class,
String.class, String.class, String.class, boolean.class, boolean.class);
}
}
@@ -17,7 +17,6 @@ import art.arcane.iris.structure.authoring.StructureOwnershipManifest;
import art.arcane.iris.structure.authoring.StructureSource; import art.arcane.iris.structure.authoring.StructureSource;
import art.arcane.iris.structure.authoring.StructureTransactionWriter; import art.arcane.iris.structure.authoring.StructureTransactionWriter;
import art.arcane.iris.structure.conversion.IrisStructureAdoptionInputKind; import art.arcane.iris.structure.conversion.IrisStructureAdoptionInputKind;
import art.arcane.iris.structure.export.VanillaJigsawExportFormat;
import art.arcane.iris.world.IrisCreator; import art.arcane.iris.world.IrisCreator;
import art.arcane.iris.generation.terrain.IrisDimension; import art.arcane.iris.generation.terrain.IrisDimension;
import art.arcane.iris.pack.value.IrisDirection; import art.arcane.iris.pack.value.IrisDirection;
@@ -413,24 +412,6 @@ public class CommandJigsawContractTest {
} }
} }
@Test
public void exportOutputIsOneSafeChildArtifact() {
Path root = Path.of("build", "jigsaw-exports").toAbsolutePath().normalize();
assertEquals(root.resolve("village.zip"), CommandJigsawExports.resolveExportDestination(
root, "village", VanillaJigsawExportFormat.ZIP));
assertEquals(root.resolve("village"), CommandJigsawExports.resolveExportDestination(
root, "village", VanillaJigsawExportFormat.DIRECTORY));
assertThrows(IllegalArgumentException.class, () -> CommandJigsawExports.resolveExportDestination(
root, "", VanillaJigsawExportFormat.DIRECTORY));
assertThrows(IllegalArgumentException.class, () -> CommandJigsawExports.resolveExportDestination(
root, ".", VanillaJigsawExportFormat.DIRECTORY));
assertThrows(IllegalArgumentException.class, () -> CommandJigsawExports.resolveExportDestination(
root, "../all-exports", VanillaJigsawExportFormat.DIRECTORY));
assertThrows(IllegalArgumentException.class, () -> CommandJigsawExports.resolveExportDestination(
root, "nested/export", VanillaJigsawExportFormat.DIRECTORY));
}
@Test @Test
public void exportLeaseRejectsDuplicatePlayerAndDestination() { public void exportLeaseRejectsDuplicatePlayerAndDestination() {
UUID firstPlayer = UUID.randomUUID(); UUID firstPlayer = UUID.randomUUID();
+1 -1
View File
@@ -62,7 +62,7 @@ apply from: file('../../gradle/java21-runtime-exclusions.gradle')
Properties rootProperties = new Properties() Properties rootProperties = new Properties()
file('../../gradle.properties').withInputStream { InputStream stream -> rootProperties.load(stream) } file('../../gradle.properties').withInputStream { InputStream stream -> rootProperties.load(stream) }
String irisVersion = providers.gradleProperty('irisVersion').getOrElse(rootProperties.getProperty('irisVersion', '4.3.6')) String irisVersion = providers.gradleProperty('irisVersion').getOrElse(rootProperties.getProperty('irisVersion', '4.3.7'))
String minecraftVersion = providers.gradleProperty('minecraftVersion').getOrElse(rootProperties.getProperty('minecraftVersion', '26.3')) String minecraftVersion = providers.gradleProperty('minecraftVersion').getOrElse(rootProperties.getProperty('minecraftVersion', '26.3'))
String fabricLoaderVersion = providers.gradleProperty('fabricLoaderVersion').getOrElse(irisModTarget.fabricLoader.toString()) String fabricLoaderVersion = providers.gradleProperty('fabricLoaderVersion').getOrElse(irisModTarget.fabricLoader.toString())
Closure<String> irisArtifactName = { String platform, String targetVersion -> Closure<String> irisArtifactName = { String platform, String targetVersion ->
+1 -1
View File
@@ -61,7 +61,7 @@ apply from: file('../../gradle/java21-runtime-exclusions.gradle')
Properties rootProperties = new Properties() Properties rootProperties = new Properties()
file('../../gradle.properties').withInputStream { InputStream stream -> rootProperties.load(stream) } file('../../gradle.properties').withInputStream { InputStream stream -> rootProperties.load(stream) }
String irisVersion = providers.gradleProperty('irisVersion').getOrElse(rootProperties.getProperty('irisVersion', '4.3.6')) String irisVersion = providers.gradleProperty('irisVersion').getOrElse(rootProperties.getProperty('irisVersion', '4.3.7'))
String minecraftVersion = providers.gradleProperty('minecraftVersion').getOrElse(rootProperties.getProperty('minecraftVersion', '26.3')) String minecraftVersion = providers.gradleProperty('minecraftVersion').getOrElse(rootProperties.getProperty('minecraftVersion', '26.3'))
String forgeVersion = providers.gradleProperty('forgeVersion').getOrElse(irisModTarget.forge.toString()) String forgeVersion = providers.gradleProperty('forgeVersion').getOrElse(irisModTarget.forge.toString())
String buildLane = gradle.extensions.extraProperties.has('irisBuildLane') String buildLane = gradle.extensions.extraProperties.has('irisBuildLane')
@@ -105,6 +105,7 @@ import java.util.Objects;
import java.util.OptionalLong; import java.util.OptionalLong;
import java.util.Set; import java.util.Set;
import java.util.TreeSet; import java.util.TreeSet;
import java.util.UUID;
import java.util.concurrent.TimeUnit; import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean; import java.util.concurrent.atomic.AtomicBoolean;
import java.util.function.IntConsumer; import java.util.function.IntConsumer;
@@ -946,6 +947,14 @@ public final class IrisModdedChunkGenerator implements NativeGeneratorOwner, Nat
+ " requires an attached generation-history runtime router.")); + " requires an attached generation-history runtime router."));
} }
private String structureStateKey() {
Engine current = boundEngineOrNull();
if (current == null || allowsGenerationHistoryBypass(current)) {
return "transient:" + UUID.randomUUID();
}
return "epoch:" + requireHistoryRouter(current, "structure state key").history().activeEpoch().epochId();
}
boolean allowsGenerationHistoryBypass(Engine current) { boolean allowsGenerationHistoryBypass(Engine current) {
if (generationMode != ModdedGenerationMode.TRANSIENT_STUDIO) { if (generationMode != ModdedGenerationMode.TRANSIENT_STUDIO) {
return false; return false;
@@ -985,7 +994,7 @@ public final class IrisModdedChunkGenerator implements NativeGeneratorOwner, Nat
ModdedBiomePolicy<H, S> policy = new ModdedBiomePolicy<>(source); ModdedBiomePolicy<H, S> policy = new ModdedBiomePolicy<>(source);
policy.bind(new ModdedBiomePolicy.RuntimeCallbacks(this::boundEngineOrNull, this::awaitStructureEngine, policy.bind(new ModdedBiomePolicy.RuntimeCallbacks(this::boundEngineOrNull, this::awaitStructureEngine,
this::allowsGenerationHistoryBypass, this::configuredStructureBiomeKeys, this::allowsGenerationHistoryBypass, this::configuredStructureBiomeKeys,
IrisModdedChunkGenerator::retainedBiomeKeys, this::visibleBiomeSeed)); IrisModdedChunkGenerator::retainedBiomeKeys, this::visibleBiomeSeed, this::structureStateKey));
return policy; return policy;
} }
@@ -1395,6 +1395,11 @@ final class ModdedBiomePolicy<H, S> implements NativeModdedBiomePolicy<H, S> {
return requireConfiguredStructureBiomeKeys(exactStructureBiomeKeys()); return requireConfiguredStructureBiomeKeys(exactStructureBiomeKeys());
} }
@Override
public String structureStateKey() {
return Objects.requireNonNull(runtime, "runtime").structureStateKey().get();
}
@Override @Override
public int horizontalSearchStep(int blockY, int searchRadius) { public int horizontalSearchStep(int blockY, int searchRadius) {
return horizontalBiomeSearchQuartStep(blockY, searchRadius); return horizontalBiomeSearchQuartStep(blockY, searchRadius);
@@ -1402,6 +1407,7 @@ final class ModdedBiomePolicy<H, S> implements NativeModdedBiomePolicy<H, S> {
record RuntimeCallbacks(Supplier<Engine> currentEngine, Supplier<Engine> awaitEngine, record RuntimeCallbacks(Supplier<Engine> currentEngine, Supplier<Engine> awaitEngine,
Predicate<Engine> historyBypass, Supplier<Set<String>> configuredStructureKeys, Predicate<Engine> historyBypass, Supplier<Set<String>> configuredStructureKeys,
Function<Engine, Set<String>> retainedBiomeKeys, LongSupplier visibleBiomeSeed) { Function<Engine, Set<String>> retainedBiomeKeys, LongSupplier visibleBiomeSeed,
Supplier<String> structureStateKey) {
} }
} }
@@ -25,6 +25,7 @@ import art.arcane.volmlib.nativelib.minecraft26_2.modded.NativeEntitySpawns;
import art.arcane.volmlib.nativelib.minecraft26_2.modded.NativeModdedLoader; import art.arcane.volmlib.nativelib.minecraft26_2.modded.NativeModdedLoader;
import art.arcane.volmlib.nativelib.minecraft26_2.modded.NativeBiomeRegistry; import art.arcane.volmlib.nativelib.minecraft26_2.modded.NativeBiomeRegistry;
import art.arcane.iris.BuildConstants; import art.arcane.iris.BuildConstants;
import art.arcane.iris.platform.ClasspathVanillaDataSource;
import art.arcane.iris.spi.IrisLogging; import art.arcane.iris.spi.IrisLogging;
import art.arcane.volmlib.nativelib.minecraft26_2.modded.NativeCommandExecutor; import art.arcane.volmlib.nativelib.minecraft26_2.modded.NativeCommandExecutor;
import art.arcane.iris.spi.PlatformGenerationRegistry; import art.arcane.iris.spi.PlatformGenerationRegistry;
@@ -35,6 +36,7 @@ import art.arcane.volmlib.nativelib.entity.NativeEntityType;
import art.arcane.iris.spi.PlatformRegistries; import art.arcane.iris.spi.PlatformRegistries;
import art.arcane.iris.spi.PlatformScheduler; import art.arcane.iris.spi.PlatformScheduler;
import art.arcane.iris.spi.PlatformStructureHooks; import art.arcane.iris.spi.PlatformStructureHooks;
import art.arcane.iris.spi.VanillaDataSource;
import art.arcane.volmlib.nativelib.terrain.NativeWorld; import art.arcane.volmlib.nativelib.terrain.NativeWorld;
import java.io.File; import java.io.File;
@@ -56,6 +58,7 @@ public final class ModdedPlatform implements IrisPlatform {
private final ModdedScheduler scheduler; private final ModdedScheduler scheduler;
private final ModdedStructureHooks structureHooks; private final ModdedStructureHooks structureHooks;
private final ModdedBiomeWriter biomeWriter; private final ModdedBiomeWriter biomeWriter;
private final ClasspathVanillaDataSource vanillaData;
public ModdedPlatform(NativeModdedLoader loader) { public ModdedPlatform(NativeModdedLoader loader) {
this.loader = loader; this.loader = loader;
@@ -65,6 +68,7 @@ public final class ModdedPlatform implements IrisPlatform {
this.scheduler = new ModdedScheduler(); this.scheduler = new ModdedScheduler();
this.structureHooks = new ModdedStructureHooks(ModdedEngineBootstrap::currentServer); this.structureHooks = new ModdedStructureHooks(ModdedEngineBootstrap::currentServer);
this.biomeWriter = new ModdedBiomeWriter(new NativeBiomeRegistry(ModdedEngineBootstrap::currentServer, "minecraft:plains")); this.biomeWriter = new ModdedBiomeWriter(new NativeBiomeRegistry(ModdedEngineBootstrap::currentServer, "minecraft:plains"));
this.vanillaData = new ClasspathVanillaDataSource(() -> overworldClass(ModdedEngineBootstrap.currentServer()));
} }
public static void errorSink(Consumer<Throwable> sink) { public static void errorSink(Consumer<Throwable> sink) {
@@ -110,6 +114,11 @@ public final class ModdedPlatform implements IrisPlatform {
return biomeWriter; return biomeWriter;
} }
@Override
public VanillaDataSource vanillaData() {
return vanillaData;
}
@Override @Override
public File dataFolder() { public File dataFolder() {
File folder = loader.configDir().resolve("iris").toFile(); File folder = loader.configDir().resolve("iris").toFile();
@@ -365,4 +374,15 @@ public final class ModdedPlatform implements IrisPlatform {
"modded-generation-registry-v1"); "modded-generation-registry-v1");
} }
static Class<?> overworldClass(NativeModdedServer server) {
if (server == null) {
throw new IllegalStateException("Vanilla data needs a running Minecraft server.");
}
NativeWorld overworld = server.overworld();
if (overworld == null) {
throw new IllegalStateException("Vanilla data needs the Minecraft server overworld to be loaded.");
}
return overworld.nativeHandle().getClass();
}
} }
@@ -189,7 +189,8 @@ final class ModdedCommandHelp {
Entry.command("install", "", ModdedHelpMessages.COMMAND_INSTALL_INSTALL_DIMENSION_TYPE_OVERRIDES_FOR_LOADED_IRIS_DIMENSIONS), Entry.command("install", "", ModdedHelpMessages.COMMAND_INSTALL_INSTALL_DIMENSION_TYPE_OVERRIDES_FOR_LOADED_IRIS_DIMENSIONS),
Entry.command("list", "", ModdedHelpMessages.COMMAND_LIST_LIST_CONFIGURED_AND_INSTALLED_DATAPACKS, "ls"), Entry.command("list", "", ModdedHelpMessages.COMMAND_LIST_LIST_CONFIGURED_AND_INSTALLED_DATAPACKS, "ls"),
Entry.command("ingest", "", ModdedHelpMessages.COMMAND_INGEST_EXPLAIN_BUKKIT_DATAPACK_INGEST_WORKFLOW, "pull"), Entry.command("ingest", "", ModdedHelpMessages.COMMAND_INGEST_EXPLAIN_BUKKIT_DATAPACK_INGEST_WORKFLOW, "pull"),
Entry.command("remove", "<id>", ModdedHelpMessages.COMMAND_REMOVE_EXPLAIN_DATAPACK_REMOVAL_WORKFLOW, "rm") Entry.command("remove", "<id>", ModdedHelpMessages.COMMAND_REMOVE_EXPLAIN_DATAPACK_REMOVAL_WORKFLOW, "rm"),
Entry.command("export", "<dimension> [version=] [mode=overworld] [namespace=iris_export] [output=] [format=zip] [objects=true] [replace=false]", ModdedHelpMessages.COMMAND_EXPORT_EXPORT_A_DIMENSION_AS_A_VANILLA_WORLDGEN_DATAPACK)
)); ));
SECTIONS.put("datapacks", SECTIONS.get("datapack")); SECTIONS.put("datapacks", SECTIONS.get("datapack"));
SECTIONS.put("dp", SECTIONS.get("datapack")); SECTIONS.put("dp", SECTIONS.get("datapack"));
@@ -18,7 +18,14 @@
package art.arcane.iris.modded.command; package art.arcane.iris.modded.command;
import art.arcane.iris.modded.ModdedEngineBootstrap;
import art.arcane.iris.modded.ModdedIrisLog; import art.arcane.iris.modded.ModdedIrisLog;
import art.arcane.iris.pack.datapack.export.WorldgenExportProgress;
import art.arcane.iris.pack.datapack.export.WorldgenExportRequest;
import art.arcane.iris.pack.datapack.export.WorldgenExportResult;
import art.arcane.iris.pack.datapack.export.WorldgenExportService;
import art.arcane.iris.pack.datapack.export.WorldgenExportServices;
import art.arcane.iris.pack.datapack.export.ir.ExportDiagnostic;
import art.arcane.iris.pack.loading.IrisData; import art.arcane.iris.pack.loading.IrisData;
import art.arcane.iris.pack.datapack.DataVersion; import art.arcane.iris.pack.datapack.DataVersion;
import art.arcane.iris.pack.PackDirectoryResolver; import art.arcane.iris.pack.PackDirectoryResolver;
@@ -33,6 +40,7 @@ import com.mojang.brigadier.context.CommandContext;
import art.arcane.volmlib.nativelib.minecraft26_2.modded.NativeCommandSource; import art.arcane.volmlib.nativelib.minecraft26_2.modded.NativeCommandSource;
import art.arcane.volmlib.nativelib.minecraft26_2.modded.NativeCommandRegistration; import art.arcane.volmlib.nativelib.minecraft26_2.modded.NativeCommandRegistration;
import art.arcane.volmlib.nativelib.minecraft26_2.modded.NativeModdedServer; import art.arcane.volmlib.nativelib.minecraft26_2.modded.NativeModdedServer;
import art.arcane.volmlib.nativelib.minecraft26_2.modded.NativeCommandText;
import art.arcane.volmlib.nativelib.terrain.NativeWorld; import art.arcane.volmlib.nativelib.terrain.NativeWorld;
import art.arcane.volmlib.nativelib.minecraft26_2.modded.NativeWorldDimensions; import art.arcane.volmlib.nativelib.minecraft26_2.modded.NativeWorldDimensions;
@@ -40,6 +48,7 @@ import java.io.File;
import java.io.IOException; import java.io.IOException;
import java.nio.charset.StandardCharsets; import java.nio.charset.StandardCharsets;
import java.nio.file.Files; import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.Collections; import java.util.Collections;
import java.util.HashSet; import java.util.HashSet;
@@ -47,15 +56,20 @@ import java.util.LinkedHashSet;
import java.util.List; import java.util.List;
import java.util.Locale; import java.util.Locale;
import java.util.Set; import java.util.Set;
import java.util.UUID;
import java.util.function.Predicate; import java.util.function.Predicate;
import art.arcane.iris.localization.DirectorCommandMessages;
import art.arcane.iris.localization.IrisLanguage; import art.arcane.iris.localization.IrisLanguage;
import art.arcane.iris.modded.localization.ModdedCommandMessages; import art.arcane.iris.modded.localization.ModdedCommandMessages;
import art.arcane.iris.localization.RuntimeUiMessages; import art.arcane.iris.localization.RuntimeUiMessages;
import art.arcane.volmlib.util.localization.LanguageAudience;
import art.arcane.volmlib.util.localization.MessageArgument; import art.arcane.volmlib.util.localization.MessageArgument;
import art.arcane.volmlib.util.localization.TextKey;
public final class ModdedDatapackCommands { public final class ModdedDatapackCommands {
private static final Predicate<NativeCommandSource> GATE = NativeCommandRegistration.GAMEMASTERS; private static final Predicate<NativeCommandSource> GATE = NativeCommandRegistration.GAMEMASTERS;
private static final String WORLD_PACK_NAME = "iris"; private static final String WORLD_PACK_NAME = "iris";
private static final int RESOURCE_REPORTS = 4;
private ModdedDatapackCommands() { private ModdedDatapackCommands() {
} }
@@ -79,6 +93,11 @@ public final class ModdedDatapackCommands {
root.then(message("ingest", "Managed Modrinth datapack ingest is Bukkit-only. On modded servers install the datapack folder or zip in world/datapacks, enable it, restart, then use /iris datapack list to confirm it is enabled. Registered structures generate natively in Iris dimensions.")); root.then(message("ingest", "Managed Modrinth datapack ingest is Bukkit-only. On modded servers install the datapack folder or zip in world/datapacks, enable it, restart, then use /iris datapack list to confirm it is enabled. Registered structures generate natively in Iris dimensions."));
root.then(message("pull", "Managed Modrinth datapack ingest is Bukkit-only. On modded servers install the datapack folder or zip in world/datapacks, enable it, restart, then use /iris datapack list to confirm it is enabled. Registered structures generate natively in Iris dimensions.")); root.then(message("pull", "Managed Modrinth datapack ingest is Bukkit-only. On modded servers install the datapack folder or zip in world/datapacks, enable it, restart, then use /iris datapack list to confirm it is enabled. Registered structures generate natively in Iris dimensions."));
root.then(NativeCommandRegistration.literal("export")
.executes(ModdedCommandTree.localized((CommandContext<NativeCommandSource> context) -> export(context.getSource(), "")))
.then(NativeCommandRegistration.argument("args", StringArgumentType.greedyString())
.executes(ModdedCommandTree.localized((CommandContext<NativeCommandSource> context) -> export(context.getSource(), StringArgumentType.getString(context, "args"))))));
root.then(message("remove", "Managed datapack removal is Bukkit-only. On modded servers disable the pack, delete its folder or zip from world/datapacks, and restart.")); root.then(message("remove", "Managed datapack removal is Bukkit-only. On modded servers disable the pack, delete its folder or zip from world/datapacks, and restart."));
root.then(message("rm", "Managed datapack removal is Bukkit-only. On modded servers disable the pack, delete its folder or zip from world/datapacks, and restart.")); root.then(message("rm", "Managed datapack removal is Bukkit-only. On modded servers disable the pack, delete its folder or zip from world/datapacks, and restart."));
@@ -291,4 +310,107 @@ public final class ModdedDatapackCommands {
} }
return null; return null;
} }
private static int export(NativeCommandSource source, String raw) {
ModdedDatapackExportArguments arguments;
try {
arguments = ModdedDatapackExportArguments.parse(raw);
} catch (IllegalArgumentException invalid) {
IrisModdedCommands.fail(source, IrisLanguage.plain(DirectorCommandMessages.COMMAND_DATAPACK_EXPORT_INVALID, MessageArgument.untrusted("reason", invalid.getMessage())));
return 0;
}
File packsRoot = ModdedPackCommands.packsRoot();
File pack = PackDirectoryResolver.resolveExisting(packsRoot, arguments.dimension());
if (pack == null) {
IrisModdedCommands.fail(source, IrisLanguage.plain(ModdedCommandMessages.MODDED_STUDIO_COMMANDS_PACK_NOT_FOUND_UNDER, MessageArgument.untrusted("name", arguments.dimension()), MessageArgument.untrusted("value", packsRoot.getAbsolutePath())));
return 0;
}
WorldgenExportService service = WorldgenExportServices.get();
WorldgenExportRequest request = arguments.toRequest(pack.toPath(), exportRoot());
IrisModdedCommands.ok(source, IrisLanguage.plain(DirectorCommandMessages.COMMAND_DATAPACK_EXPORT_STARTED, MessageArgument.untrusted("dimension", request.dimension()), MessageArgument.untrusted("version", arguments.target().getVersion())));
ExportFeedback feedback = new ExportFeedback(source);
Thread worker = new Thread(() -> runExport(service, request, feedback), "Iris Datapack Export");
worker.setDaemon(true);
worker.start();
return 1;
}
private static Path exportRoot() {
return ModdedEngineBootstrap.loader().configDir().resolve("irisworldgen").resolve("exports").resolve("datapacks");
}
private static void runExport(WorldgenExportService service, WorldgenExportRequest request, ExportFeedback feedback) {
WorldgenExportResult result;
try {
result = service.export(request, feedback);
} catch (Throwable failure) {
ModdedIrisLog.error("Iris datapack export failed for " + request.dimension(), failure);
feedback.post(() -> IrisModdedCommands.fail(feedback.source, IrisLanguage.plain(DirectorCommandMessages.COMMAND_DATAPACK_EXPORT_FAILED, MessageArgument.untrusted("reason", IrisLanguage.errorDetail(failure)))));
return;
}
feedback.post(() -> reportExport(feedback.source, result));
}
private static void reportExport(NativeCommandSource source, WorldgenExportResult result) {
for (ExportDiagnostic diagnostic : result.diagnostics()) {
String line = IrisLanguage.plain(DirectorCommandMessages.COMMAND_DATAPACK_EXPORT_DIAGNOSTIC, MessageArgument.untrusted("key", diagnosticLabel(diagnostic)), MessageArgument.untrusted("message", diagnostic.message()));
ModdedCommandFeedback.send(source, NativeCommandText.literal(line).withStyle(diagnosticFormat(diagnostic.severity())));
}
if (result.isSuccess()) {
IrisModdedCommands.ok(source, IrisLanguage.plain(DirectorCommandMessages.COMMAND_DATAPACK_EXPORT_EXPORTED, MessageArgument.untrusted("path", result.output())));
return;
}
TextKey outcome = result.status() == WorldgenExportResult.Status.REJECTED
? DirectorCommandMessages.COMMAND_DATAPACK_EXPORT_REJECTED
: DirectorCommandMessages.COMMAND_DATAPACK_EXPORT_FAILED;
IrisModdedCommands.fail(source, IrisLanguage.plain(outcome, MessageArgument.untrusted("reason", result.message())));
}
private static NativeCommandText.Format diagnosticFormat(ExportDiagnostic.Severity severity) {
return switch (severity) {
case ERROR -> NativeCommandText.Format.RED;
case WARNING -> NativeCommandText.Format.YELLOW;
case INFO -> NativeCommandText.Format.GRAY;
};
}
private static String diagnosticLabel(ExportDiagnostic diagnostic) {
return diagnostic.resource().isEmpty()
? diagnostic.limit().name()
: diagnostic.limit().name() + " " + diagnostic.resource();
}
private static final class ExportFeedback implements WorldgenExportProgress {
private final NativeCommandSource source;
private final NativeModdedServer server;
private final UUID audience;
private ExportFeedback(NativeCommandSource source) {
this.source = source;
this.server = source.server();
this.audience = source.playerId();
}
@Override
public void stage(String stage) {
post(() -> progress(IrisLanguage.plain(DirectorCommandMessages.COMMAND_DATAPACK_EXPORT_STAGE, MessageArgument.untrusted("stage", stage))));
}
@Override
public void resources(int done, int total) {
int stride = Math.max(1, total / RESOURCE_REPORTS);
if (done != total && done % stride != 0) {
return;
}
post(() -> progress(IrisLanguage.plain(DirectorCommandMessages.COMMAND_DATAPACK_EXPORT_RESOURCES, MessageArgument.untrusted("done", done), MessageArgument.untrusted("total", total))));
}
private void post(Runnable task) {
server.execute(() -> LanguageAudience.run(audience, task));
}
private void progress(String message) {
ModdedCommandFeedback.send(source, NativeCommandText.literal(message).withStyle(NativeCommandText.Format.GRAY));
}
}
} }
@@ -0,0 +1,128 @@
/*
* Iris is a World Generator for Minecraft Servers
* Copyright (c) 2026 Arcane Arts (Volmit Software)
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
package art.arcane.iris.modded.command;
import art.arcane.iris.pack.datapack.DataVersion;
import art.arcane.iris.pack.datapack.export.DatapackExportDestinations;
import art.arcane.iris.pack.datapack.export.WorldgenExportMode;
import art.arcane.iris.pack.datapack.export.WorldgenExportRequest;
import art.arcane.iris.pack.datapack.export.WorldgenExportTargets;
import java.nio.file.Path;
import java.util.HashMap;
import java.util.Map;
import java.util.Objects;
import java.util.Set;
import java.util.regex.Pattern;
record ModdedDatapackExportArguments(String dimension, String version, String mode, String namespace, String output,
String format, boolean objects, boolean replace) {
private static final Pattern DIMENSION = Pattern.compile("[A-Za-z0-9_/.-]+");
private static final String VERSION_KEY = "version";
private static final String MODE_KEY = "mode";
private static final String NAMESPACE_KEY = "namespace";
private static final String OUTPUT_KEY = "output";
private static final String FORMAT_KEY = "format";
private static final String OBJECTS_KEY = "objects";
private static final String REPLACE_KEY = "replace";
private static final Set<String> OPTIONS = Set.of(VERSION_KEY, MODE_KEY, NAMESPACE_KEY, OUTPUT_KEY, FORMAT_KEY,
OBJECTS_KEY, REPLACE_KEY);
ModdedDatapackExportArguments {
Objects.requireNonNull(dimension, "dimension");
Objects.requireNonNull(version, "version");
Objects.requireNonNull(mode, "mode");
Objects.requireNonNull(namespace, "namespace");
Objects.requireNonNull(output, "output");
Objects.requireNonNull(format, "format");
}
static ModdedDatapackExportArguments parse(String raw) {
String trimmed = raw == null ? "" : raw.trim();
String[] tokens = trimmed.isEmpty() ? new String[0] : trimmed.split("\\s+");
String dimension = null;
Map<String, String> options = new HashMap<>(OPTIONS.size());
for (String token : tokens) {
int equals = token.indexOf('=');
if (equals < 0) {
if (dimension != null) {
throw new IllegalArgumentException("unexpected argument '" + token + "'");
}
dimension = token;
continue;
}
String key = token.substring(0, equals);
if (!OPTIONS.contains(key)) {
throw new IllegalArgumentException("unknown option '" + key + "'");
}
if (options.putIfAbsent(key, token.substring(equals + 1)) != null) {
throw new IllegalArgumentException("option '" + key + "' given twice");
}
}
if (dimension == null) {
throw new IllegalArgumentException("missing dimension");
}
if (!DIMENSION.matcher(dimension).matches() || dimension.contains("..")) {
throw new IllegalArgumentException("invalid dimension key '" + dimension + "'");
}
ModdedDatapackExportArguments arguments = new ModdedDatapackExportArguments(
dimension,
options.getOrDefault(VERSION_KEY, ""),
options.getOrDefault(MODE_KEY, WorldgenExportRequest.DEFAULT_MODE),
options.getOrDefault(NAMESPACE_KEY, WorldgenExportRequest.DEFAULT_NAMESPACE),
options.getOrDefault(OUTPUT_KEY, ""),
options.getOrDefault(FORMAT_KEY, WorldgenExportRequest.DEFAULT_FORMAT),
flag(OBJECTS_KEY, options.get(OBJECTS_KEY), true),
flag(REPLACE_KEY, options.get(REPLACE_KEY), false));
arguments.target();
WorldgenExportMode.parse(arguments.mode());
DatapackExportDestinations.parseFormat(arguments.format());
return arguments;
}
DataVersion target() {
return WorldgenExportTargets.resolve(version);
}
WorldgenExportRequest toRequest(Path pack, Path exportRoot) {
return WorldgenExportRequest.builder()
.pack(pack)
.dimension(dimension)
.version(version)
.mode(mode)
.namespace(namespace)
.output(output)
.format(format)
.objects(objects)
.replace(replace)
.exportRoot(exportRoot)
.build();
}
private static boolean flag(String key, String raw, boolean fallback) {
if (raw == null) {
return fallback;
}
return switch (raw) {
case "true" -> true;
case "false" -> false;
default -> throw new IllegalArgumentException(key + " must be true or false");
};
}
}
@@ -348,6 +348,11 @@ public final class ModdedPregenMethod implements PregeneratorMethod {
return false; return false;
} }
@Override
public void onRegionBounds(int minRegionX, int minRegionZ, int maxRegionX, int maxRegionZ) {
hydrology.bound(minRegionX, minRegionZ, maxRegionX, maxRegionZ);
}
@Override @Override
public void onRegionsAhead(List<long[]> regions) { public void onRegionsAhead(List<long[]> regions) {
if (!closing.get()) { if (!closing.get()) {
@@ -330,6 +330,10 @@ public final class ModdedHelpMessages {
"iris.modded.help.entry.command.remove", "iris.modded.help.entry.command.remove",
"Explain datapack removal workflow" "Explain datapack removal workflow"
); );
public static final TextKey COMMAND_EXPORT_EXPORT_A_DIMENSION_AS_A_VANILLA_WORLDGEN_DATAPACK = TextKey.of(
"iris.modded.help.entry.command.export_datapack",
"Export a dimension as a vanilla worldgen datapack"
);
public static final TextKey COMMAND_LIST_REGENERATE_STRUCTURE_INDEX_JSON = TextKey.of( public static final TextKey COMMAND_LIST_REGENERATE_STRUCTURE_INDEX_JSON = TextKey.of(
"iris.modded.help.entry.command.list_structures", "iris.modded.help.entry.command.list_structures",
"Regenerate structure-index.json" "Regenerate structure-index.json"
@@ -505,6 +509,7 @@ public final class ModdedHelpMessages {
COMMAND_LIST_LIST_CONFIGURED_AND_INSTALLED_DATAPACKS, COMMAND_LIST_LIST_CONFIGURED_AND_INSTALLED_DATAPACKS,
COMMAND_INGEST_EXPLAIN_BUKKIT_DATAPACK_INGEST_WORKFLOW, COMMAND_INGEST_EXPLAIN_BUKKIT_DATAPACK_INGEST_WORKFLOW,
COMMAND_REMOVE_EXPLAIN_DATAPACK_REMOVAL_WORKFLOW, COMMAND_REMOVE_EXPLAIN_DATAPACK_REMOVAL_WORKFLOW,
COMMAND_EXPORT_EXPORT_A_DIMENSION_AS_A_VANILLA_WORLDGEN_DATAPACK,
COMMAND_LIST_REGENERATE_STRUCTURE_INDEX_JSON, COMMAND_LIST_REGENERATE_STRUCTURE_INDEX_JSON,
COMMAND_INFO_RESOLVE_AN_IRIS_STRUCTURE_GRAPH_AND_REPORT_BOUNDS, COMMAND_INFO_RESOLVE_AN_IRIS_STRUCTURE_GRAPH_AND_REPORT_BOUNDS,
COMMAND_PLACE_ASSEMBLE_AND_PLACE_AN_IRIS_STRUCTURE_AT_YOUR_LOCATION, COMMAND_PLACE_ASSEMBLE_AND_PLACE_AN_IRIS_STRUCTURE_AT_YOUR_LOCATION,
@@ -102,6 +102,15 @@ public class ModdedBiomePolicyTest {
assertSame(desert, policy.orderedPossibleBiomes().getFirst()); assertSame(desert, policy.orderedPossibleBiomes().getFirst());
} }
@Test
public void structureStateKeyComesFromTheBoundRuntime() {
ModdedBiomePolicy<BiomeToken, Object> policy = new ModdedBiomePolicy<>(new Access(new ArrayList<>()));
policy.bind(new ModdedBiomePolicy.RuntimeCallbacks(() -> null, () -> null, engine -> false,
Set::of, engine -> Set.of(), () -> 0L, () -> "epoch:example"));
assertEquals("epoch:example", policy.structureStateKey());
}
@Test @Test
public void missingRequiredStructureBiomesFailBeforePublishingAPartialSet() { public void missingRequiredStructureBiomesFailBeforePublishingAPartialSet() {
AtomicReference<Set<String>> configured = new AtomicReference<>(Set.of("example:missing")); AtomicReference<Set<String>> configured = new AtomicReference<>(Set.of("example:missing"));
@@ -130,7 +139,7 @@ public class ModdedBiomePolicyTest {
ModdedBiomePolicy<BiomeToken, Object> policy = new ModdedBiomePolicy<>( ModdedBiomePolicy<BiomeToken, Object> policy = new ModdedBiomePolicy<>(
new Access(new ArrayList<>(List.of(forest)))); new Access(new ArrayList<>(List.of(forest))));
policy.bind(new ModdedBiomePolicy.RuntimeCallbacks(() -> engine, () -> engine, bound -> true, policy.bind(new ModdedBiomePolicy.RuntimeCallbacks(() -> engine, () -> engine, bound -> true,
() -> Set.of(forest.key()), bound -> Set.of(), () -> 0L)); () -> Set.of(forest.key()), bound -> Set.of(), () -> 0L, () -> "transient:test"));
assertSame(forest, policy.requiredStructureBiome(6000, 0, -6000, null)); assertSame(forest, policy.requiredStructureBiome(6000, 0, -6000, null));
assertSame(forest, policy.requiredStructureBiome(6000, -30, -6000, null)); assertSame(forest, policy.requiredStructureBiome(6000, -30, -6000, null));
@@ -212,7 +221,7 @@ public class ModdedBiomePolicyTest {
ModdedBiomePolicy<BiomeToken, Object> policy = new ModdedBiomePolicy<>( ModdedBiomePolicy<BiomeToken, Object> policy = new ModdedBiomePolicy<>(
new Access(new ArrayList<>(List.of(plains, mirage)))); new Access(new ArrayList<>(List.of(plains, mirage))));
policy.bind(new ModdedBiomePolicy.RuntimeCallbacks(() -> engine, () -> engine, bound -> false, policy.bind(new ModdedBiomePolicy.RuntimeCallbacks(() -> engine, () -> engine, bound -> false,
() -> Set.of(plains.key(), mirage.key()), bound -> Set.of(), () -> 0L)); () -> Set.of(plains.key(), mirage.key()), bound -> Set.of(), () -> 0L, () -> "transient:test"));
int[] columns = {0, 1024, 2048, 3072, 4096, 5120}; int[] columns = {0, 1024, 2048, 3072, 4096, 5120};
NativeBiomeSourcePolicy.BiomeLocation<BiomeToken> found = policy.findClosestBiome3d( NativeBiomeSourcePolicy.BiomeLocation<BiomeToken> found = policy.findClosestBiome3d(
@@ -241,7 +250,7 @@ public class ModdedBiomePolicyTest {
private static ModdedBiomePolicy<BiomeToken, Object> boundTo(Engine engine) { private static ModdedBiomePolicy<BiomeToken, Object> boundTo(Engine engine) {
ModdedBiomePolicy<BiomeToken, Object> policy = new ModdedBiomePolicy<>(new Access(new ArrayList<>())); ModdedBiomePolicy<BiomeToken, Object> policy = new ModdedBiomePolicy<>(new Access(new ArrayList<>()));
policy.bind(new ModdedBiomePolicy.RuntimeCallbacks(() -> engine, () -> engine, bound -> true, policy.bind(new ModdedBiomePolicy.RuntimeCallbacks(() -> engine, () -> engine, bound -> true,
Set::of, bound -> Set.of(), () -> 0L)); Set::of, bound -> Set.of(), () -> 0L, () -> "transient:test"));
return policy; return policy;
} }
@@ -249,7 +258,7 @@ public class ModdedBiomePolicyTest {
AtomicReference<Set<String>> configured) { AtomicReference<Set<String>> configured) {
ModdedBiomePolicy<BiomeToken, Object> policy = new ModdedBiomePolicy<>(new Access(entries)); ModdedBiomePolicy<BiomeToken, Object> policy = new ModdedBiomePolicy<>(new Access(entries));
policy.bind(new ModdedBiomePolicy.RuntimeCallbacks(() -> null, () -> null, engine -> false, policy.bind(new ModdedBiomePolicy.RuntimeCallbacks(() -> null, () -> null, engine -> false,
configured::get, engine -> Set.of(), () -> 0L)); configured::get, engine -> Set.of(), () -> 0L, () -> "transient:test"));
return policy; return policy;
} }
@@ -0,0 +1,50 @@
package art.arcane.iris.modded;
import art.arcane.iris.platform.ClasspathVanillaDataSource;
import art.arcane.volmlib.nativelib.minecraft26_2.modded.NativeModdedServer;
import art.arcane.volmlib.nativelib.terrain.NativeWorld;
import net.minecraft.SharedConstants;
import net.minecraft.server.Bootstrap;
import net.minecraft.server.level.ServerLevel;
import org.junit.BeforeClass;
import org.junit.Test;
import java.nio.charset.StandardCharsets;
import static org.junit.Assert.assertSame;
import static org.junit.Assert.assertThrows;
import static org.junit.Assert.assertTrue;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
public class ModdedVanillaDataTest {
private static final String PLAINS = "data/minecraft/worldgen/biome/plains.json";
private static final String BIOME_TAGS = "data/minecraft/tags/worldgen/biome";
@BeforeClass
public static void bootstrapRegistries() {
SharedConstants.tryDetectVersion();
Bootstrap.bootStrap();
}
@Test
public void overworldLevelClassReadsTheGameVanillaData() {
NativeModdedServer server = mock(NativeModdedServer.class);
NativeWorld overworld = mock(NativeWorld.class);
when(overworld.nativeHandle()).thenReturn(mock(ServerLevel.class));
when(server.overworld()).thenReturn(overworld);
Class<?> anchor = ModdedPlatform.overworldClass(server);
assertSame(ServerLevel.class, anchor);
ClasspathVanillaDataSource source = new ClasspathVanillaDataSource(() -> anchor);
String plains = new String(source.read(PLAINS).orElseThrow(), StandardCharsets.UTF_8);
assertTrue(plains.contains("\"temperature\""));
assertTrue(source.list(BIOME_TAGS).contains(BIOME_TAGS + "/is_overworld.json"));
}
@Test
public void vanillaDataNeedsALoadedOverworld() {
NativeModdedServer server = mock(NativeModdedServer.class);
assertThrows(IllegalStateException.class, () -> ModdedPlatform.overworldClass(null));
assertThrows(IllegalStateException.class, () -> ModdedPlatform.overworldClass(server));
}
}
@@ -63,6 +63,9 @@ public class IrisModdedCommandParityTest {
CommandNode<CommandSourceStack> studio = child(iris, "studio"); CommandNode<CommandSourceStack> studio = child(iris, "studio");
child(studio, "package"); child(studio, "package");
child(studio, "pkg"); child(studio, "pkg");
CommandNode<CommandSourceStack> datapackExport = child(child(iris, "datapack"), "export");
assertNotNull(datapackExport.getCommand());
assertNotNull(child(datapackExport, "args").getCommand());
CommandNode<CommandSourceStack> download = child(iris, "download"); CommandNode<CommandSourceStack> download = child(iris, "download");
CommandNode<CommandSourceStack> source = child(download, "source"); CommandNode<CommandSourceStack> source = child(download, "source");
@@ -211,6 +214,8 @@ public class IrisModdedCommandParityTest {
assertTrue(root.stream().anyMatch((String line) -> line.startsWith("/iris teleport <dimension> [player] (tp) - "))); assertTrue(root.stream().anyMatch((String line) -> line.startsWith("/iris teleport <dimension> [player] (tp) - ")));
assertTrue(ModdedCommandHelp.consoleLines("pregen").stream().anyMatch((String line) -> assertTrue(ModdedCommandHelp.consoleLines("pregen").stream().anyMatch((String line) ->
line.startsWith("/iris pregen start <radius> [dimension] [at] [x] [z] [gui] [sync] [nocache] - "))); line.startsWith("/iris pregen start <radius> [dimension] [at] [x] [z] [gui] [sync] [nocache] - ")));
assertTrue(ModdedCommandHelp.consoleLines("datapack").stream().anyMatch((String line) ->
line.startsWith("/iris datapack export <dimension> [version=] [mode=overworld] [namespace=iris_export] [output=] [format=zip] [objects=true] [replace=false] - ")));
} }
private static CommandNode<CommandSourceStack> child( private static CommandNode<CommandSourceStack> child(
@@ -0,0 +1,87 @@
package art.arcane.iris.modded.command;
import art.arcane.iris.pack.datapack.DataVersion;
import art.arcane.iris.pack.datapack.export.WorldgenExportRequest;
import org.junit.Test;
import java.nio.file.Path;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertSame;
import static org.junit.Assert.assertThrows;
import static org.junit.Assert.assertTrue;
public class ModdedDatapackExportArgumentsTest {
@Test
public void unknownKeyFails() {
assertEquals("unknown option 'seed'", reason("overworld seed=1"));
assertEquals("unknown option 'Version'", reason("overworld Version=26.3"));
}
@Test
public void duplicateKeyFails() {
assertEquals("option 'replace' given twice", reason("overworld replace=true replace=false"));
assertEquals("option 'version' given twice", reason("overworld version= version=26.3"));
}
@Test
public void firstBareTokenIsDimension() {
ModdedDatapackExportArguments arguments = ModdedDatapackExportArguments.parse(
" version=26.3 overworld mode=dimension namespace=worlds output=export_one format=directory objects=false replace=true ");
assertEquals("overworld", arguments.dimension());
assertEquals("26.3", arguments.version());
assertEquals("dimension", arguments.mode());
assertEquals("worlds", arguments.namespace());
assertEquals("export_one", arguments.output());
assertEquals("directory", arguments.format());
assertFalse(arguments.objects());
assertTrue(arguments.replace());
assertSame(DataVersion.V26_3, arguments.target());
assertEquals("unexpected argument 'underworld'", reason("overworld underworld"));
assertEquals("missing dimension", reason(" "));
assertEquals("missing dimension", reason("version=26.3"));
}
@Test
public void defaultsMatchBukkit() {
ModdedDatapackExportArguments arguments = ModdedDatapackExportArguments.parse("overworld");
assertEquals("", arguments.version());
assertEquals("overworld", arguments.mode());
assertEquals("iris_export", arguments.namespace());
assertEquals("", arguments.output());
assertEquals("zip", arguments.format());
assertTrue(arguments.objects());
assertFalse(arguments.replace());
assertSame(DataVersion.V26_3, arguments.target());
Path pack = Path.of("packs", "overworld");
Path exportRoot = Path.of("exports", "datapacks");
WorldgenExportRequest expected = WorldgenExportRequest.builder()
.pack(pack)
.dimension("overworld")
.exportRoot(exportRoot)
.build();
assertEquals(expected, arguments.toRequest(pack, exportRoot));
}
@Test
public void unsupportedValuesFailWithSharedMessages() {
assertEquals("Unsupported datapack export version '1.21.1'. Supported: 26.3",
reason("overworld version=1.21.1"));
assertEquals("Unsupported datapack export version '26.2'. Supported: 26.3",
reason("overworld version=26.2"));
assertEquals("Unknown export mode 'nether'. Expected: preset, dimension, overworld",
reason("overworld mode=nether"));
assertEquals("Export format must be directory or zip.", reason("overworld format=rar"));
assertEquals("objects must be true or false", reason("overworld objects=yes"));
assertEquals("replace must be true or false", reason("overworld replace=1"));
assertEquals("invalid dimension key 'a..b'", reason("a..b"));
}
private static String reason(String raw) {
return assertThrows(IllegalArgumentException.class, () -> ModdedDatapackExportArguments.parse(raw)).getMessage();
}
}
@@ -0,0 +1,64 @@
package art.arcane.volmlib.nativelib.minecraft26_2.modded;
import art.arcane.volmlib.nativelib.terrain.NativeModdedBiomePolicy;
import com.google.gson.JsonElement;
import com.mojang.serialization.DataResult;
import com.mojang.serialization.JsonOps;
import com.mojang.serialization.MapCodec;
import net.minecraft.SharedConstants;
import net.minecraft.core.Holder;
import net.minecraft.server.Bootstrap;
import net.minecraft.world.level.biome.Biome;
import net.minecraft.world.level.biome.BiomeSource;
import net.minecraft.world.level.biome.Climate;
import org.junit.BeforeClass;
import org.junit.Test;
import java.lang.reflect.Constructor;
import java.util.Set;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
public class NativeModdedBiomeSourceCodecTest {
@BeforeClass
public static void bootstrap() {
SharedConstants.tryDetectVersion();
Bootstrap.bootStrap();
}
@Test
public void runtimeSourceEncodeReportsErrorInsteadOfThrowing() {
DataResult<JsonElement> result = BiomeSource.CODEC.encodeStart(JsonOps.INSTANCE, runtimeSource("epoch:unused"));
assertTrue(result.isError());
}
@Test
@SuppressWarnings("unchecked")
public void structureStateSourceEncodesItsGenerationKey() throws ReflectiveOperationException {
BiomeSource structureState = structureState(runtimeSource("epoch:example"));
MapCodec<BiomeSource> codec = (MapCodec<BiomeSource>) NativeModdedBiomeSource.STRUCTURE_STATE_CODEC;
JsonElement encoded = codec.codec().encodeStart(JsonOps.INSTANCE, structureState).getOrThrow();
assertEquals("epoch:example", encoded.getAsJsonObject().get("generation").getAsString());
assertTrue(codec.codec().parse(JsonOps.INSTANCE, encoded).isError());
}
private static BiomeSource structureState(NativeModdedBiomeSource<?> source) throws ReflectiveOperationException {
Class<?> type = Class.forName(NativeModdedBiomeSource.class.getName() + "$StructureStateBiomeSource");
Constructor<?> constructor = type.getDeclaredConstructor(NativeModdedBiomeSource.class, Set.class);
constructor.setAccessible(true);
return (BiomeSource) constructor.newInstance(source, Set.<Holder<Biome>>of());
}
@SuppressWarnings("unchecked")
private static NativeModdedBiomeSource<NativeModdedBiomePolicy<Holder<Biome>, Climate.Sampler>> runtimeSource(String key) {
NativeModdedBiomePolicy<Holder<Biome>, Climate.Sampler> policy = mock(NativeModdedBiomePolicy.class);
when(policy.structureStateKey()).thenReturn(key);
return new NativeModdedBiomeSource<>(new NativeModdedBiomeSource.Options<>(
mock(BiomeSource.class), () -> null, access -> policy));
}
}
@@ -0,0 +1,104 @@
package art.arcane.volmlib.nativelib.minecraft26_2.modded;
import art.arcane.volmlib.nativelib.terrain.NativeBlockState;
import com.mojang.serialization.Codec;
import net.minecraft.SharedConstants;
import net.minecraft.core.BlockPos;
import net.minecraft.core.Holder;
import net.minecraft.core.IdMapper;
import net.minecraft.server.Bootstrap;
import net.minecraft.world.level.ChunkPos;
import net.minecraft.world.level.LevelHeightAccessor;
import net.minecraft.world.level.biome.Biome;
import net.minecraft.world.level.biome.BiomeGenerationSettings;
import net.minecraft.world.level.biome.BiomeSpecialEffects;
import net.minecraft.world.level.biome.MobSpawnSettings;
import net.minecraft.world.level.block.Block;
import net.minecraft.world.level.block.Blocks;
import net.minecraft.world.level.block.state.BlockState;
import net.minecraft.world.level.chunk.ChunkAccess;
import net.minecraft.world.level.chunk.PalettedContainer;
import net.minecraft.world.level.chunk.PalettedContainerFactory;
import net.minecraft.world.level.chunk.PalettedContainerRO;
import net.minecraft.world.level.chunk.ProtoChunk;
import net.minecraft.world.level.chunk.Strategy;
import net.minecraft.world.level.chunk.UpgradeData;
import org.junit.Assume;
import org.junit.BeforeClass;
import org.junit.Test;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.TimeUnit;
import static org.junit.Assert.assertSame;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
public class NativeModdedChunkGeneratorTerrainWriteTest {
private static final int MIN_Y = -64;
private static final int HEIGHT = 384;
private static final int SURFACE = 10;
@BeforeClass
public static void bootstrap() {
SharedConstants.tryDetectVersion();
Bootstrap.bootStrap();
}
@Test
public void freshSectionsInstallWithoutSelfLockingTheirContainers() throws Exception {
BlockState stone = Blocks.STONE.defaultBlockState();
Assume.assumeTrue("block-state ids are not baked on this classpath",
Block.BLOCK_STATE_REGISTRY.getId(stone) >= 0);
NativeBlockState nativeStone = mock(NativeBlockState.class);
when(nativeStone.nativeHandle()).thenReturn(stone);
NativeModdedGeneratorPolicy.BlockBuffer blocks = (x, y, z) -> y + MIN_Y < SURFACE ? nativeStone : null;
ProtoChunk chunk = newChunk();
Method writeBlocks = NativeModdedChunkGenerator.class.getDeclaredMethod("writeBlocks", ChunkAccess.class,
NativeModdedGeneratorPolicy.BlockBuffer.class, int.class, int.class);
writeBlocks.setAccessible(true);
CompletableFuture.runAsync(() -> invoke(writeBlocks, chunk, blocks)).get(30, TimeUnit.SECONDS);
assertSame(stone, chunk.getBlockState(new BlockPos(3, SURFACE - 1, 5)));
assertSame(stone, chunk.getBlockState(new BlockPos(0, MIN_Y, 0)));
assertSame(Blocks.AIR.defaultBlockState(), chunk.getBlockState(new BlockPos(3, SURFACE, 5)));
}
private static void invoke(Method writeBlocks, ChunkAccess chunk, NativeModdedGeneratorPolicy.BlockBuffer blocks) {
try {
writeBlocks.invoke(null, chunk, blocks, MIN_Y, HEIGHT);
} catch (IllegalAccessException | InvocationTargetException failure) {
throw new IllegalStateException(failure);
}
}
private static ProtoChunk newChunk() {
return new ProtoChunk(new ChunkPos(0, 0), UpgradeData.EMPTY, LevelHeightAccessor.create(MIN_Y, HEIGHT),
palettedContainerFactory(), null);
}
private static PalettedContainerFactory palettedContainerFactory() {
Strategy<BlockState> blockStrategy = Strategy.createForBlockStates(Block.BLOCK_STATE_REGISTRY);
Codec<PalettedContainer<BlockState>> blockCodec = PalettedContainer.codecRW(
BlockState.CODEC, blockStrategy, Blocks.AIR.defaultBlockState());
Biome biome = new Biome.BiomeBuilder()
.hasPrecipitation(false)
.temperature(0.8F)
.downfall(0.4F)
.specialEffects(new BiomeSpecialEffects.Builder().waterColor(0x3F76E4).build())
.mobSpawnSettings(MobSpawnSettings.EMPTY)
.generationSettings(BiomeGenerationSettings.EMPTY)
.build();
Holder<Biome> biomeHolder = Holder.direct(biome);
IdMapper<Holder<Biome>> biomeIds = new IdMapper<>(1);
biomeIds.add(biomeHolder);
Strategy<Holder<Biome>> biomeStrategy = Strategy.createForBiomes(biomeIds);
Codec<PalettedContainerRO<Holder<Biome>>> biomeCodec = PalettedContainer.codecRO(
Biome.CODEC, biomeStrategy, biomeHolder);
return new PalettedContainerFactory(blockStrategy, Blocks.AIR.defaultBlockState(), blockCodec,
biomeStrategy, biomeHolder, biomeCodec);
}
}
@@ -0,0 +1,98 @@
package art.arcane.volmlib.nativelib.minecraft26_2.modded;
import art.arcane.volmlib.nativelib.terrain.NativeBlockState;
import com.mojang.serialization.Lifecycle;
import net.minecraft.SharedConstants;
import net.minecraft.core.BlockPos;
import net.minecraft.core.MappedRegistry;
import net.minecraft.core.RegistrationInfo;
import net.minecraft.core.Registry;
import net.minecraft.core.registries.Registries;
import net.minecraft.resources.ResourceKey;
import net.minecraft.resources.ResourceLocation;
import net.minecraft.server.Bootstrap;
import net.minecraft.world.level.ChunkPos;
import net.minecraft.world.level.LevelHeightAccessor;
import net.minecraft.world.level.biome.Biome;
import net.minecraft.world.level.biome.BiomeGenerationSettings;
import net.minecraft.world.level.biome.BiomeSpecialEffects;
import net.minecraft.world.level.biome.MobSpawnSettings;
import net.minecraft.world.level.block.Block;
import net.minecraft.world.level.block.Blocks;
import net.minecraft.world.level.block.state.BlockState;
import net.minecraft.world.level.chunk.ChunkAccess;
import net.minecraft.world.level.chunk.ProtoChunk;
import net.minecraft.world.level.chunk.UpgradeData;
import org.junit.Assume;
import org.junit.BeforeClass;
import org.junit.Test;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.TimeUnit;
import static org.junit.Assert.assertSame;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
public class NativeModdedChunkGeneratorTerrainWriteTest {
private static final int MIN_Y = -64;
private static final int HEIGHT = 384;
private static final int SURFACE = 10;
@BeforeClass
public static void bootstrap() {
SharedConstants.tryDetectVersion();
Bootstrap.bootStrap();
}
@Test
public void freshSectionsInstallWithoutSelfLockingTheirContainers() throws Exception {
BlockState stone = Blocks.STONE.defaultBlockState();
Assume.assumeTrue("block-state ids are not baked on this classpath",
Block.BLOCK_STATE_REGISTRY.getId(stone) >= 0);
NativeBlockState nativeStone = mock(NativeBlockState.class);
when(nativeStone.nativeHandle()).thenReturn(stone);
NativeModdedGeneratorPolicy.BlockBuffer blocks = (x, y, z) -> y + MIN_Y < SURFACE ? nativeStone : null;
ProtoChunk chunk = newChunk();
Method writeBlocks = NativeModdedChunkGenerator.class.getDeclaredMethod("writeBlocks", ChunkAccess.class,
NativeModdedGeneratorPolicy.BlockBuffer.class, int.class, int.class);
writeBlocks.setAccessible(true);
CompletableFuture.runAsync(() -> invoke(writeBlocks, chunk, blocks)).get(30, TimeUnit.SECONDS);
assertSame(stone, chunk.getBlockState(new BlockPos(3, SURFACE - 1, 5)));
assertSame(stone, chunk.getBlockState(new BlockPos(0, MIN_Y, 0)));
assertSame(Blocks.AIR.defaultBlockState(), chunk.getBlockState(new BlockPos(3, SURFACE, 5)));
}
private static void invoke(Method writeBlocks, ChunkAccess chunk, NativeModdedGeneratorPolicy.BlockBuffer blocks) {
try {
writeBlocks.invoke(null, chunk, blocks, MIN_Y, HEIGHT);
} catch (IllegalAccessException | InvocationTargetException failure) {
throw new IllegalStateException(failure);
}
}
private static ProtoChunk newChunk() {
return new ProtoChunk(new ChunkPos(0, 0), UpgradeData.EMPTY, LevelHeightAccessor.create(MIN_Y, HEIGHT),
biomeRegistry(), null);
}
private static Registry<Biome> biomeRegistry() {
Biome biome = new Biome.BiomeBuilder()
.hasPrecipitation(false)
.temperature(0.8F)
.downfall(0.4F)
.specialEffects(new BiomeSpecialEffects.Builder().fogColor(0).waterColor(0x3F76E4)
.waterFogColor(0).skyColor(0).build())
.mobSpawnSettings(MobSpawnSettings.EMPTY)
.generationSettings(BiomeGenerationSettings.EMPTY)
.build();
MappedRegistry<Biome> registry = new MappedRegistry<>(Registries.BIOME, Lifecycle.stable());
registry.register(ResourceKey.create(Registries.BIOME, ResourceLocation.withDefaultNamespace("plains")),
biome, RegistrationInfo.BUILT_IN);
return registry.freeze();
}
}
+1 -1
View File
@@ -60,7 +60,7 @@ apply from: file('../../gradle/java21-runtime-exclusions.gradle')
Properties rootProperties = new Properties() Properties rootProperties = new Properties()
file('../../gradle.properties').withInputStream { InputStream stream -> rootProperties.load(stream) } file('../../gradle.properties').withInputStream { InputStream stream -> rootProperties.load(stream) }
String irisVersion = providers.gradleProperty('irisVersion').getOrElse(rootProperties.getProperty('irisVersion', '4.3.6')) String irisVersion = providers.gradleProperty('irisVersion').getOrElse(rootProperties.getProperty('irisVersion', '4.3.7'))
String minecraftVersion = providers.gradleProperty('minecraftVersion').getOrElse(rootProperties.getProperty('minecraftVersion', '26.3')) String minecraftVersion = providers.gradleProperty('minecraftVersion').getOrElse(rootProperties.getProperty('minecraftVersion', '26.3'))
String neoForgeVersion = providers.gradleProperty('neoForgeVersion').getOrElse(irisModTarget.neoForge.toString()) String neoForgeVersion = providers.gradleProperty('neoForgeVersion').getOrElse(irisModTarget.neoForge.toString())
Closure<String> irisArtifactName = { String platform, String targetVersion -> Closure<String> irisArtifactName = { String platform, String targetVersion ->
+58 -1
View File
@@ -55,7 +55,7 @@ apply from: file('gradle/runtime-relocations.gradle')
apply from: file('gradle/volmlib-stamp.gradle') apply from: file('gradle/volmlib-stamp.gradle')
group = 'art.arcane' group = 'art.arcane'
version = providers.gradleProperty('irisVersion').getOrElse('4.3.6') version = providers.gradleProperty('irisVersion').getOrElse('4.3.7')
project(':core') { project(':core') {
group = 'art.arcane' group = 'art.arcane'
@@ -821,6 +821,13 @@ project(':adapters:bukkit:nms:v26_3_R1') {
} }
sourceSets.test.compileClasspath += sourceSets.offline.output sourceSets.test.compileClasspath += sourceSets.offline.output
sourceSets.test.runtimeClasspath += sourceSets.offline.output sourceSets.test.runtimeClasspath += sourceSets.offline.output
tasks.named('test').configure {
String exportProbePack = providers.gradleProperty('iris.exportProbePack').orNull
if (exportProbePack != null) {
systemProperty('iris.exportProbePack', exportProbePack)
maxHeapSize = '6g'
}
}
TaskProvider<Jar> offlineJar = tasks.register('offlineJar', Jar) { TaskProvider<Jar> offlineJar = tasks.register('offlineJar', Jar) {
dependsOn(offlineClasses) dependsOn(offlineClasses)
@@ -848,6 +855,30 @@ project(':adapters:bukkit:nms:v26_3_R1') {
} }
} }
tasks.register('datapackCostProbe', JavaExec) {
group = 'verification'
description = 'Exports an Iris dimension as a 26.3 worldgen datapack and measures its load, chunk and density function cost against vanilla.'
dependsOn(offlineClasses)
mainClass = 'art.arcane.iris.probe.DatapackCostProbe'
classpath = sourceSets.offline.runtimeClasspath
maxHeapSize = providers.gradleProperty('probeHeap').getOrElse('24G')
jvmArgs('--add-modules', 'jdk.incubator.vector', '-Djava.awt.headless=true')
doFirst {
String dimension = providers.gradleProperty('probeDimension').getOrElse('overworld')
String output = providers.gradleProperty('probeOutput')
.getOrElse(layout.buildDirectory.file("reports/datapack-cost/${dimension}.json").get().asFile.absolutePath)
args([rootProject.file(providers.gradleProperty('probePack').get()).absolutePath, dimension,
providers.gradleProperty('probeSeed').getOrElse('1337'),
providers.gradleProperty('probeChunkSide').getOrElse('6'),
providers.gradleProperty('probeRounds').getOrElse('4'),
providers.gradleProperty('probeWarmupSide').getOrElse('6'),
providers.gradleProperty('probeCalls').getOrElse('256'),
providers.gradleProperty('probeCallSeconds').getOrElse('60'),
rootProject.file(output).absolutePath,
layout.buildDirectory.dir('tmp/datapack-cost').get().asFile.absolutePath])
}
}
tasks.register('regionMantleInspection', JavaExec) { tasks.register('regionMantleInspection', JavaExec) {
group = 'verification' group = 'verification'
dependsOn(testClasses) dependsOn(testClasses)
@@ -877,6 +908,32 @@ project(':adapters:bukkit:nms:v26_3_R1') {
} }
} }
tasks.register('datapackFidelityProbe', JavaExec) {
group = 'verification'
description = 'Exports an Iris dimension as a 26.3 worldgen datapack and compares both generators on the same seed.'
dependsOn(offlineClasses)
mainClass = 'art.arcane.iris.probe.DatapackFidelityProbe'
classpath = sourceSets.offline.runtimeClasspath
maxHeapSize = '8G'
jvmArgs('--add-modules', 'jdk.incubator.vector', '-Djava.awt.headless=true')
doFirst {
String dimension = providers.gradleProperty('probeDimension').get()
String seed = providers.gradleProperty('probeSeed').get()
String mode = providers.gradleProperty('probeMode').getOrElse('full')
boolean biomes = mode == 'biomes'
String output = providers.gradleProperty('probeOutput')
.getOrElse(layout.buildDirectory.file("reports/datapack-fidelity/${dimension}-${seed}${biomes ? '-biomes' : ''}.json").get().asFile.absolutePath)
args([rootProject.file(providers.gradleProperty('probePack').get()).absolutePath, dimension, seed,
biomes ? providers.gradleProperty('probeRadiusChunks').getOrElse('0') : providers.gradleProperty('probeRadiusChunks').get(),
biomes ? providers.gradleProperty('probeStride').getOrElse('1') : providers.gradleProperty('probeStride').get(),
rootProject.file(output).absolutePath,
providers.gradleProperty('probeSurveyBlocks').getOrElse('65536'),
providers.gradleProperty('probeSurveyStride').getOrElse('256'),
providers.gradleProperty('probeSurveySeeds').getOrElse(biomes ? '7' : '1'),
mode])
}
}
} }
+4 -5
View File
@@ -193,6 +193,10 @@ tasks.named('test').configure {
maxHeapSize = '2g' maxHeapSize = '2g'
jvmArgs('-XX:+ExitOnOutOfMemoryError') jvmArgs('-XX:+ExitOnOutOfMemoryError')
systemProperty('iris.generation.counters', 'true') systemProperty('iris.generation.counters', 'true')
String exportProbePack = providers.gradleProperty('iris.exportProbePack').orNull
if (exportProbePack != null) {
systemProperty('iris.exportProbePack', exportProbePack)
}
} }
java { java {
@@ -360,14 +364,9 @@ List<String> unusedVolmLibPackages = [
] ]
List<String> unusedVolmLibClasses = [ List<String> unusedVolmLibClasses = [
'art/arcane/volmlib/util/nbt/io/ParseException',
'art/arcane/volmlib/util/nbt/io/SNBTDeserializer',
'art/arcane/volmlib/util/nbt/io/SNBTParser',
'art/arcane/volmlib/util/nbt/io/SNBTSerializer', 'art/arcane/volmlib/util/nbt/io/SNBTSerializer',
'art/arcane/volmlib/util/nbt/io/SNBTUtil', 'art/arcane/volmlib/util/nbt/io/SNBTUtil',
'art/arcane/volmlib/util/nbt/io/SNBTWriter', 'art/arcane/volmlib/util/nbt/io/SNBTWriter',
'art/arcane/volmlib/util/nbt/io/StringPointer',
'art/arcane/volmlib/util/io/StringDeserializer',
'art/arcane/volmlib/util/io/StringSerializer', 'art/arcane/volmlib/util/io/StringSerializer',
'art/arcane/volmlib/util/json/EnumType', 'art/arcane/volmlib/util/json/EnumType',
'art/arcane/volmlib/util/json/HTTP', 'art/arcane/volmlib/util/json/HTTP',
+2
View File
@@ -119,6 +119,8 @@ art/arcane/iris/studio/StudioSVC.java
art/arcane/iris/studio/edit/BlockSignal.java art/arcane/iris/studio/edit/BlockSignal.java
art/arcane/iris/studio/edit/DustRevealer.java art/arcane/iris/studio/edit/DustRevealer.java
art/arcane/iris/studio/generation/JigsawStudioGenerator.java art/arcane/iris/studio/generation/JigsawStudioGenerator.java
art/arcane/iris/studio/jigsaw/JigsawStudioConnectorTool.java
art/arcane/iris/studio/jigsaw/JigsawStudioFloorPainter.java
art/arcane/iris/studio/generation/ObjectStudioGenerator.java art/arcane/iris/studio/generation/ObjectStudioGenerator.java
art/arcane/iris/studio/jigsaw/JigsawStudioAutosaveScheduler.java art/arcane/iris/studio/jigsaw/JigsawStudioAutosaveScheduler.java
art/arcane/iris/studio/jigsaw/JigsawStudioCapture.java art/arcane/iris/studio/jigsaw/JigsawStudioCapture.java
@@ -20,7 +20,7 @@ package art.arcane.iris.generation.chunk;
import art.arcane.volmlib.util.hunk.Hunk; import art.arcane.volmlib.util.hunk.Hunk;
public final class ColumnExtentListeningHunk<T> implements Hunk<T>, ColumnExtent, ColumnRunHunk<T> { public final class ColumnExtentListeningHunk<T> implements Hunk<T>, ColumnExtent, ColumnRunHunk<T>, ColumnReadHunk<T> {
private final Hunk<T> source; private final Hunk<T> source;
private final Listener<T> listener; private final Listener<T> listener;
@@ -46,6 +46,11 @@ public final class ColumnExtentListeningHunk<T> implements Hunk<T>, ColumnExtent
return source.getRaw(x, y, z); return source.getRaw(x, y, z);
} }
@Override
public void readColumn(int x, int z, int yFromInclusive, int yToInclusive, T[] target, int targetOffset) {
ColumnReadHunk.readColumn(source, x, z, yFromInclusive, yToInclusive, target, targetOffset);
}
@Override @Override
public int getWidth() { public int getWidth() {
return source.getWidth(); return source.getWidth();
@@ -0,0 +1,37 @@
/*
* Iris is a World Generator for Minecraft Bukkit Servers
* Copyright (c) 2026 Arcane Arts (Volmit Software)
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
package art.arcane.iris.generation.chunk;
import art.arcane.volmlib.util.hunk.Hunk;
public interface ColumnReadHunk<T> {
void readColumn(int x, int z, int yFromInclusive, int yToInclusive, T[] target, int targetOffset);
@SuppressWarnings("unchecked")
static <T> void readColumn(Hunk<T> hunk, int x, int z, int yFromInclusive, int yToInclusive, T[] target,
int targetOffset) {
if (hunk instanceof ColumnReadHunk<?> reads) {
((ColumnReadHunk<T>) reads).readColumn(x, z, yFromInclusive, yToInclusive, target, targetOffset);
return;
}
for (int y = yFromInclusive; y <= yToInclusive; y++) {
target[targetOffset + y - yFromInclusive] = hunk.getRaw(x, y, z);
}
}
}
@@ -27,7 +27,8 @@ import java.lang.invoke.MethodHandles;
import java.lang.invoke.VarHandle; import java.lang.invoke.VarHandle;
import java.util.Arrays; import java.util.Arrays;
public final class SectionBuffer implements Hunk<NativeBlockState>, NativeSectionVolume, ColumnExtent, ColumnRunHunk<NativeBlockState> { public final class SectionBuffer implements Hunk<NativeBlockState>, NativeSectionVolume, ColumnExtent, ColumnRunHunk<NativeBlockState>,
ColumnReadHunk<NativeBlockState> {
private static final BoundBlockState AIR = BoundBlockState.of("AIR"); private static final BoundBlockState AIR = BoundBlockState.of("AIR");
private static final VarHandle SECTIONS = MethodHandles.arrayElementVarHandle(short[][].class); private static final VarHandle SECTIONS = MethodHandles.arrayElementVarHandle(short[][].class);
private static final VarHandle COLUMN_TOPS = MethodHandles.arrayElementVarHandle(int[].class); private static final VarHandle COLUMN_TOPS = MethodHandles.arrayElementVarHandle(int[].class);
@@ -43,6 +44,8 @@ public final class SectionBuffer implements Hunk<NativeBlockState>, NativeSectio
private PaletteEntry[] slots; private PaletteEntry[] slots;
private NativeBlockState[] states; private NativeBlockState[] states;
private int paletteSize; private int paletteSize;
private NativeBlockState air;
private PaletteEntry lastEntry;
public SectionBuffer(int height) { public SectionBuffer(int height) {
if (height <= 0) { if (height <= 0) {
@@ -76,12 +79,17 @@ public final class SectionBuffer implements Hunk<NativeBlockState>, NativeSectio
if (state == null) { if (state == null) {
return 0; return 0;
} }
PaletteEntry last = lastEntry;
if (last != null && last.state == state) {
return last.id;
}
PaletteEntry[] table = slots; PaletteEntry[] table = slots;
int mask = table.length - 1; int mask = table.length - 1;
int slot = System.identityHashCode(state) & mask; int slot = System.identityHashCode(state) & mask;
PaletteEntry entry; PaletteEntry entry;
while ((entry = table[slot]) != null) { while ((entry = table[slot]) != null) {
if (entry.state == state) { if (entry.state == state) {
lastEntry = entry;
return entry.id; return entry.id;
} }
slot = (slot + 1) & mask; slot = (slot + 1) & mask;
@@ -101,7 +109,7 @@ public final class SectionBuffer implements Hunk<NativeBlockState>, NativeSectio
@Override @Override
public NativeBlockState getRaw(int x, int y, int z) { public NativeBlockState getRaw(int x, int y, int z) {
if (y < 0 || y >= height) { if (y < 0 || y >= height) {
return AIR.get(); return air();
} }
short[] section = sections[y >> 4]; short[] section = sections[y >> 4];
if (section != null) { if (section != null) {
@@ -110,7 +118,7 @@ public final class SectionBuffer implements Hunk<NativeBlockState>, NativeSectio
return state(id); return state(id);
} }
} }
return AIR.get(); return air();
} }
@Override @Override
@@ -126,6 +134,32 @@ public final class SectionBuffer implements Hunk<NativeBlockState>, NativeSectio
return id == 0 ? null : state(id); return id == 0 ? null : state(id);
} }
@Override
public void readColumn(int x, int z, int yFromInclusive, int yToInclusive, NativeBlockState[] target, int targetOffset) {
int column = (z << 4) | x;
int shift = targetOffset - yFromInclusive;
int y = yFromInclusive;
while (y <= yToInclusive) {
if (y < 0 || y >= height) {
target[shift + y] = air();
y++;
continue;
}
int sectionIndex = y >> 4;
int last = Math.min(Math.min(yToInclusive, height - 1), (sectionIndex << 4) | 15);
short[] section = sections[sectionIndex];
if (section == null) {
Arrays.fill(target, shift + y, shift + last + 1, air());
} else {
for (int row = y; row <= last; row++) {
int id = section[((row & 15) << 8) | column] & 0xFFFF;
target[shift + row] = id == 0 ? air() : state(id);
}
}
y = last + 1;
}
}
@Override @Override
public void fillColumnRun(int x, int z, int yFromInclusive, int yToInclusive, NativeBlockState value) { public void fillColumnRun(int x, int z, int yFromInclusive, int yToInclusive, NativeBlockState value) {
if (value == null || yToInclusive < yFromInclusive) { if (value == null || yToInclusive < yFromInclusive) {
@@ -176,6 +210,15 @@ public final class SectionBuffer implements Hunk<NativeBlockState>, NativeSectio
} }
} }
private NativeBlockState air() {
NativeBlockState resolved = air;
if (resolved == null) {
resolved = AIR.get();
air = resolved;
}
return resolved;
}
private short[] writableSection(int sectionIndex) { private short[] writableSection(int sectionIndex) {
short[] section = sections[sectionIndex]; short[] section = sections[sectionIndex];
if (section != null) { if (section != null) {
@@ -28,9 +28,14 @@ import art.arcane.iris.generation.terrain.IrisDimension;
import art.arcane.volmlib.util.collection.KMap; import art.arcane.volmlib.util.collection.KMap;
import java.util.Objects; import java.util.Objects;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicReferenceArray;
public final class IrisContext { public final class IrisContext {
private static final ThreadLocal<IrisContext> CONTEXT = new ThreadLocal<>(); private static final int SLOTS = 512;
private static final AtomicReferenceArray<Binding> BINDINGS = new AtomicReferenceArray<>(SLOTS);
private static final ThreadLocal<Binding> OVERFLOW = new ThreadLocal<>();
private static final AtomicInteger OVERFLOW_BINDINGS = new AtomicInteger();
private final Engine engine; private final Engine engine;
private final ChunkContext chunkContext; private final ChunkContext chunkContext;
private final long generationSessionId; private final long generationSessionId;
@@ -42,7 +47,8 @@ public final class IrisContext {
} }
public static IrisContext get() { public static IrisContext get() {
return CONTEXT.get(); Binding binding = binding(Thread.currentThread());
return binding == null ? null : binding.context;
} }
public static IrisContext require() { public static IrisContext require() {
@@ -55,9 +61,10 @@ public final class IrisContext {
public static Scope open(Engine engine, long generationSessionId, ChunkContext chunkContext) { public static Scope open(Engine engine, long generationSessionId, ChunkContext chunkContext) {
Thread thread = Thread.currentThread(); Thread thread = Thread.currentThread();
IrisContext previous = CONTEXT.get(); Binding binding = bind(thread);
IrisContext previous = binding.context;
IrisContext installed = new IrisContext(engine, generationSessionId, chunkContext); IrisContext installed = new IrisContext(engine, generationSessionId, chunkContext);
CONTEXT.set(installed); binding.context = installed;
return new Scope(thread, previous, installed); return new Scope(thread, previous, installed);
} }
@@ -99,6 +106,42 @@ public final class IrisContext {
.qput("queued", mantle.getUnloadRegionCount())); .qput("queued", mantle.getUnloadRegionCount()));
} }
private static Binding binding(Thread thread) {
Binding binding = BINDINGS.getPlain(slot(thread));
if (binding != null && binding.owner == thread) {
return binding;
}
return OVERFLOW_BINDINGS.get() == 0 ? null : OVERFLOW.get();
}
private static Binding bind(Thread thread) {
Binding binding = binding(thread);
if (binding != null) {
return binding;
}
binding = new Binding(thread);
if (!BINDINGS.compareAndSet(slot(thread), null, binding)) {
binding.overflow = true;
OVERFLOW.set(binding);
OVERFLOW_BINDINGS.incrementAndGet();
}
return binding;
}
private static void release(Binding binding) {
binding.context = null;
if (binding.overflow) {
OVERFLOW.remove();
OVERFLOW_BINDINGS.decrementAndGet();
return;
}
BINDINGS.compareAndSet(slot(binding.owner), binding, null);
}
private static int slot(Thread thread) {
return (int) thread.threadId() & (SLOTS - 1);
}
public static final class Scope implements NativeGenerationScope { public static final class Scope implements NativeGenerationScope {
private final Thread owner; private final Thread owner;
private final IrisContext previous; private final IrisContext previous;
@@ -119,16 +162,26 @@ public final class IrisContext {
if (Thread.currentThread() != owner) { if (Thread.currentThread() != owner) {
throw new IllegalStateException("Iris execution context scope closed from a different thread."); throw new IllegalStateException("Iris execution context scope closed from a different thread.");
} }
IrisContext current = CONTEXT.get(); Binding binding = binding(owner);
if (current != installed) { if (binding == null || binding.context != installed) {
throw new IllegalStateException("Iris execution context scopes must close in LIFO order."); throw new IllegalStateException("Iris execution context scopes must close in LIFO order.");
} }
if (previous == null) { if (previous == null) {
CONTEXT.remove(); release(binding);
} else { } else {
CONTEXT.set(previous); binding.context = previous;
} }
closed = true; closed = true;
} }
} }
private static final class Binding {
private final Thread owner;
private boolean overflow;
private IrisContext context;
private Binding(Thread owner) {
this.owner = owner;
}
}
} }
@@ -22,6 +22,9 @@ public class IrisHydrology {
@Description("Fraction of eligible water features to keep after local placement probabilities. 0.5 keeps half the candidate river sources, ponds, lakes and underground water features without changing their configured sizes.") @Description("Fraction of eligible water features to keep after local placement probabilities. 0.5 keeps half the candidate river sources, ponds, lakes and underground water features without changing their configured sizes.")
private double density = 1D; private double density = 1D;
@Description("Wet and dry zones that decide where streams and ponds gather. Within a wet zone the configured density applies unchanged.")
private IrisHydrologyClimate climate = new IrisHydrologyClimate();
@Description("How water drains across the terrain before rivers and ponds are placed.") @Description("How water drains across the terrain before rivers and ponds are placed.")
private IrisHydrologyDrainage drainage = new IrisHydrologyDrainage(); private IrisHydrologyDrainage drainage = new IrisHydrologyDrainage();
@@ -0,0 +1,37 @@
package art.arcane.iris.generation.hydrology.config;
import art.arcane.iris.generation.noise.IrisGeneratorStyle;
import art.arcane.iris.generation.noise.NoiseStyle;
import art.arcane.iris.pack.schema.annotation.MaxNumber;
import art.arcane.iris.pack.schema.annotation.MinNumber;
import art.arcane.volmlib.util.documentation.Description;
import lombok.Data;
import lombok.NoArgsConstructor;
import lombok.experimental.Accessors;
@Accessors(chain = true)
@NoArgsConstructor
@Description("Wet and dry zones across the dimension. A noise field splits the land into zones: wet zones keep the rainfall and pond density of their biome, dry zones lose them. Streams and ponds then cluster in the wet zones instead of reaching every hill, and a river that rises in a wet zone still flows on through dry land.")
@Data
public class IrisHydrologyClimate {
@Description("Turns wet and dry zones on. When off, rainfall and pond density follow the hydrology policies alone.")
private boolean enabled = false;
@Description("Noise that lays out the zones. Land where its value is below dryness is dry. A larger zoom makes larger zones.")
private IrisGeneratorStyle style = NoiseStyle.SIMPLEX.style().zoomed(24D);
@MinNumber(0)
@MaxNumber(1)
@Description("Noise value below which land is dry. 0 keeps all land wet, and higher values dry out more of it.")
private double dryness = 0.5D;
@MinNumber(0)
@MaxNumber(1)
@Description("Width of the noise band over which rainfall fades from dry to wet. 0 gives sharp zone edges.")
private double transition = 0.1D;
@MinNumber(0)
@MaxNumber(1)
@Description("Share of the area's rainfall and pond density kept in dry zones. 0 lets dry zones start no streams or ponds of their own.")
private double dryRainfall = 0D;
}
@@ -24,12 +24,12 @@ public class IrisHydrologyDeformation {
@MinNumber(0) @MinNumber(0)
@MaxNumber(16) @MaxNumber(16)
@Description("How much steeper banks become on steep terrain.") @Description("How much steeper a bank stays where the natural ground between it and the water is steep.")
private double bankSteepFactor = 3.5D; private double bankSteepFactor = 3.5D;
@MinNumber(0) @MinNumber(0)
@MaxNumber(2) @MaxNumber(2)
@Description("Strength of the noise that roughens bank edges.") @Description("Strength of the smooth noise that roughens bank edges.")
private double bankNoise = 0.2D; private double bankNoise = 0.2D;
@Description("Keeps caves from opening under water, so rivers and ponds never drain into them.") @Description("Keeps caves from opening under water, so rivers and ponds never drain into them.")
@@ -31,13 +31,16 @@ public class IrisHydrologyPolicy {
@Description("Multiplier on how many ponds form in this area. With no value set anywhere it is 1.") @Description("Multiplier on how many ponds form in this area. With no value set anywhere it is 1.")
private Double pondDensity = null; private Double pondDensity = null;
@Description("Whether the dimension's wet and dry zones apply in this area. false keeps this area's full rainfall and pond density even inside a dry zone. With no value set anywhere they apply.")
private Boolean climateZones = null;
@Description("Whether streams and rivers may form in this area. With no value set anywhere they are allowed.") @Description("Whether streams and rivers may form in this area. With no value set anywhere they are allowed.")
private IrisHydrologyPermission streams = null; private IrisHydrologyPermission streams = null;
@Description("Whether ponds may form in this area. With no value set anywhere they are allowed.") @Description("Whether ponds may form in this area. With no value set anywhere they are allowed.")
private IrisHydrologyPermission ponds = null; private IrisHydrologyPermission ponds = null;
@Description("Name of a fluid profile that fills ponds and lakes whose floor lies in this area, instead of the ponds' own fluid. Channels always use the fluid of their tier.") @Description("Name of a fluid profile that fills the water in this area instead of its usual fluid: ponds and lakes whose floor lies here, and every stretch of stream or river that flows through here.")
private String fluid = null; private String fluid = null;
@RegistryListResource(IrisBiome.class) @RegistryListResource(IrisBiome.class)
@@ -1,53 +0,0 @@
package art.arcane.iris.generation.hydrology.engine;
import art.arcane.iris.generation.hydrology.grid.HeightLattice;
import art.arcane.iris.generation.hydrology.grid.HydrologySampleBuffer;
import art.arcane.iris.generation.hydrology.grid.HydrologyTerrainSource;
import java.util.Objects;
final class FineLatticeSource implements HydrologyTerrainSource {
private final HeightLattice fine;
private final HydrologyTerrainSource terrain;
FineLatticeSource(HeightLattice fine, HydrologyTerrainSource terrain) {
this.fine = Objects.requireNonNull(fine, "fine");
this.terrain = Objects.requireNonNull(terrain, "terrain");
}
@Override
public void sample(int[] xs, int[] zs, HydrologySampleBuffer out, boolean baseOnly) {
if (baseOnly) {
throw new IllegalArgumentException("The fine lattice holds natural heights only");
}
if (xs.length != zs.length) {
throw new IllegalArgumentException("Fine lattice read has " + xs.length + " x and " + zs.length + " z coordinates");
}
int cell = fine.cell();
int[] cellXs = new int[xs.length];
int[] cellZs = new int[zs.length];
for (int index = 0; index < xs.length; index++) {
if (Math.floorMod(xs[index], cell) != 0 || Math.floorMod(zs[index], cell) != 0) {
throw new IllegalArgumentException("Column " + xs[index] + "," + zs[index] + " is not a node of the " + cell + "-block fine lattice");
}
cellXs[index] = Math.floorDiv(xs[index], cell);
cellZs[index] = Math.floorDiv(zs[index], cell);
}
fine.fill(cellXs, cellZs, out);
}
@Override
public int seaLevel() {
return terrain.seaLevel();
}
@Override
public int minY() {
return terrain.minY();
}
@Override
public int maxY() {
return terrain.maxY();
}
}
@@ -11,23 +11,21 @@ import art.arcane.iris.generation.hydrology.water.ReachShaper;
import java.util.Objects; import java.util.Objects;
public record HydrologyPlanners(RegionPlanner regions, WindowPlanner windows, HeightLattice windowLattice) { public record HydrologyPlanners(RegionPlanner regions, WindowPlanner windows, HeightLattice fineLattice) {
public HydrologyPlanners { public HydrologyPlanners {
Objects.requireNonNull(regions, "regions"); Objects.requireNonNull(regions, "regions");
Objects.requireNonNull(windows, "windows"); Objects.requireNonNull(windows, "windows");
Objects.requireNonNull(windowLattice, "windowLattice"); Objects.requireNonNull(fineLattice, "fineLattice");
} }
public static HydrologyPlanners create(HydrologySettings settings, HydrologyTerrainSource source) { public static HydrologyPlanners create(HydrologySettings settings, HydrologyTerrainSource source) {
Objects.requireNonNull(settings, "settings"); Objects.requireNonNull(settings, "settings");
Objects.requireNonNull(source, "source"); Objects.requireNonNull(source, "source");
HeightLattice fine = new HeightLattice(source, WindowPlanner.FINE_CELL, false, HydrologyPlanCache.FINE_LATTICE_CELLS); HeightLattice fine = new HeightLattice(source, WindowPlanner.FINE_CELL, false, HydrologyPlanCache.FINE_LATTICE_CELLS);
HeightLattice window = new HeightLattice(new FineLatticeSource(fine, source), WindowPlanner.CELL, false,
HydrologyPlanCache.WINDOW_LATTICE_CELLS);
PondPlanner ponds = new PondPlanner(settings); PondPlanner ponds = new PondPlanner(settings);
ReachShaper shaper = new ReachShaper(settings); ReachShaper shaper = new ReachShaper(settings);
RegionPlanner regions = new RegionPlanner(settings, RegionPlanner regions = new RegionPlanner(settings,
new HeightLattice(source, RegionPlanner.CELL, true, HydrologyPlanCache.REGION_LATTICE_CELLS), fine, ponds, shaper); new HeightLattice(source, RegionPlanner.CELL, true, HydrologyPlanCache.REGION_LATTICE_CELLS), fine, ponds, shaper);
return new HydrologyPlanners(regions, new WindowPlanner(settings, window, fine, ponds, shaper), window); return new HydrologyPlanners(regions, new WindowPlanner(settings, fine, ponds, shaper), fine);
} }
} }
@@ -18,6 +18,12 @@ public final class HydrologyPregeneration implements AutoCloseable {
} }
} }
public synchronized void bound(int minRegionX, int minRegionZ, int maxRegionX, int maxRegionZ) {
if (!closed) {
runtime.boundPregeneration(epoch, minRegionX, minRegionZ, maxRegionX, maxRegionZ);
}
}
@Override @Override
public synchronized void close() { public synchronized void close() {
if (!closed) { if (!closed) {
@@ -14,8 +14,10 @@ import art.arcane.iris.generation.hydrology.plan.WindowKey;
import art.arcane.iris.generation.hydrology.query.HydrologyFeatureQuery; import art.arcane.iris.generation.hydrology.query.HydrologyFeatureQuery;
import art.arcane.iris.generation.hydrology.query.HydrologyFeatureRef; import art.arcane.iris.generation.hydrology.query.HydrologyFeatureRef;
import art.arcane.iris.generation.hydrology.query.HydrologyFeatureType; import art.arcane.iris.generation.hydrology.query.HydrologyFeatureType;
import art.arcane.iris.generation.hydrology.raster.ChannelFluids;
import art.arcane.iris.generation.hydrology.raster.ChunkRasterizer; import art.arcane.iris.generation.hydrology.raster.ChunkRasterizer;
import art.arcane.iris.generation.hydrology.raster.ChunkWaterRaster; import art.arcane.iris.generation.hydrology.raster.ChunkWaterRaster;
import art.arcane.iris.generation.hydrology.raster.NaturalHeights;
import art.arcane.iris.generation.hydrology.underground.SeaInlet; import art.arcane.iris.generation.hydrology.underground.SeaInlet;
import art.arcane.iris.generation.hydrology.underground.SeaInletRasterizer; import art.arcane.iris.generation.hydrology.underground.SeaInletRasterizer;
import art.arcane.iris.generation.hydrology.water.HydrologyTier; import art.arcane.iris.generation.hydrology.water.HydrologyTier;
@@ -36,6 +38,7 @@ import java.io.UncheckedIOException;
import java.nio.file.Path; import java.nio.file.Path;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.Collection; import java.util.Collection;
import java.util.HashMap;
import java.util.HashSet; import java.util.HashSet;
import java.util.LinkedHashMap; import java.util.LinkedHashMap;
import java.util.List; import java.util.List;
@@ -50,13 +53,12 @@ import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicLong; import java.util.concurrent.atomic.AtomicLong;
import java.util.function.BooleanSupplier; import java.util.function.BooleanSupplier;
import java.util.function.Function; import java.util.function.Function;
import java.util.function.IntBinaryOperator;
import java.util.function.LongSupplier; import java.util.function.LongSupplier;
import java.util.function.Predicate; import java.util.function.Predicate;
import java.util.function.Supplier; import java.util.function.Supplier;
public final class HydrologyRuntime { public final class HydrologyRuntime {
private static final int RASTER_CAPACITY = 1024; private static final int RASTER_CAPACITY = 8192;
private static final int KEEP_SCOPES = 2; private static final int KEEP_SCOPES = 2;
private static final int CHUNKS_PER_WINDOW = WindowKey.SIZE >> 4; private static final int CHUNKS_PER_WINDOW = WindowKey.SIZE >> 4;
public static final int MAX_PREGENERATION_REGIONS = 4096; public static final int MAX_PREGENERATION_REGIONS = 4096;
@@ -67,16 +69,18 @@ public final class HydrologyRuntime {
private final HydrologySettings settings; private final HydrologySettings settings;
private final HydrologyPlanCache cache; private final HydrologyPlanCache cache;
private final ChunkRasterizer rasterizer; private final ChunkRasterizer rasterizer;
private final IntBinaryOperator naturalHeight; private final NaturalHeights naturalHeight;
private final BooleanSupplier mainThread; private final BooleanSupplier mainThread;
private final RasterLru rasters; private final RasterLru rasters;
private final ThreadLocal<LastRaster> last; private final ThreadLocal<LastRaster> last;
private final Set<Long> queued; private final Set<Long> queued;
private final ConcurrentHashMap<Long, CompletableFuture<ChunkWaterRaster>> building;
private final RasterFailures failures; private final RasterFailures failures;
private final RasterFailures windowPreviewFailures; private final RasterFailures windowPreviewFailures;
private final RasterFailures regionPreviewFailures; private final RasterFailures regionPreviewFailures;
private final Set<Object> previews; private final Set<Object> previews;
private final LinkedHashMap<WindowKey, WindowPrefetch> prefetches; private final LinkedHashMap<WindowKey, WindowPrefetch> prefetches;
private final Map<Object, PregenArea> pregenAreas;
private final HydrologyBinding binding; private final HydrologyBinding binding;
private final Function<PreparedHydrologyCacheIdentity, HydrologyPlanStore> stores; private final Function<PreparedHydrologyCacheIdentity, HydrologyPlanStore> stores;
private final boolean caveFluidPolicies; private final boolean caveFluidPolicies;
@@ -94,37 +98,41 @@ public final class HydrologyRuntime {
this.rasters = null; this.rasters = null;
this.last = null; this.last = null;
this.queued = null; this.queued = null;
this.building = null;
this.failures = null; this.failures = null;
this.windowPreviewFailures = null; this.windowPreviewFailures = null;
this.regionPreviewFailures = null; this.regionPreviewFailures = null;
this.previews = null; this.previews = null;
this.prefetches = null; this.prefetches = null;
this.pregenAreas = null;
this.binding = HydrologyBinding.UNBOUND; this.binding = HydrologyBinding.UNBOUND;
this.stores = identity -> null; this.stores = identity -> null;
this.caveFluidPolicies = false; this.caveFluidPolicies = false;
} }
public HydrologyRuntime(HydrologySettings settings, HydrologyPlanCache cache, IntBinaryOperator naturalHeight, public HydrologyRuntime(HydrologySettings settings, HydrologyPlanCache cache, NaturalHeights naturalHeight,
BooleanSupplier mainThread, LongSupplier ticker) { BooleanSupplier mainThread, LongSupplier ticker) {
this(settings, cache, naturalHeight, mainThread, ticker, HydrologyBinding.UNBOUND, identity -> null, false); this(settings, cache, naturalHeight, ChannelFluids.TIER, mainThread, ticker, HydrologyBinding.UNBOUND, identity -> null, false);
} }
private HydrologyRuntime(HydrologySettings settings, HydrologyPlanCache cache, IntBinaryOperator naturalHeight, private HydrologyRuntime(HydrologySettings settings, HydrologyPlanCache cache, NaturalHeights naturalHeight,
BooleanSupplier mainThread, LongSupplier ticker, HydrologyBinding binding, ChannelFluids channelFluids, BooleanSupplier mainThread, LongSupplier ticker, HydrologyBinding binding,
Function<PreparedHydrologyCacheIdentity, HydrologyPlanStore> stores, boolean caveFluidPolicies) { Function<PreparedHydrologyCacheIdentity, HydrologyPlanStore> stores, boolean caveFluidPolicies) {
this.settings = Objects.requireNonNull(settings, "settings"); this.settings = Objects.requireNonNull(settings, "settings");
this.cache = Objects.requireNonNull(cache, "cache"); this.cache = Objects.requireNonNull(cache, "cache");
this.rasterizer = new ChunkRasterizer(settings); this.rasterizer = new ChunkRasterizer(settings, channelFluids);
this.naturalHeight = Objects.requireNonNull(naturalHeight, "naturalHeight"); this.naturalHeight = Objects.requireNonNull(naturalHeight, "naturalHeight");
this.mainThread = Objects.requireNonNull(mainThread, "mainThread"); this.mainThread = Objects.requireNonNull(mainThread, "mainThread");
this.rasters = new RasterLru(); this.rasters = new RasterLru();
this.last = ThreadLocal.withInitial(LastRaster::new); this.last = ThreadLocal.withInitial(LastRaster::new);
this.queued = ConcurrentHashMap.newKeySet(); this.queued = ConcurrentHashMap.newKeySet();
this.building = new ConcurrentHashMap<>();
this.failures = new RasterFailures(ticker); this.failures = new RasterFailures(ticker);
this.windowPreviewFailures = new RasterFailures(ticker); this.windowPreviewFailures = new RasterFailures(ticker);
this.regionPreviewFailures = new RasterFailures(ticker); this.regionPreviewFailures = new RasterFailures(ticker);
this.previews = new HashSet<>(); this.previews = new HashSet<>();
this.prefetches = new LinkedHashMap<>(PREFETCH_CAPACITY * 2, 0.75f, true); this.prefetches = new LinkedHashMap<>(PREFETCH_CAPACITY * 2, 0.75f, true);
this.pregenAreas = new HashMap<>();
this.binding = Objects.requireNonNull(binding, "binding"); this.binding = Objects.requireNonNull(binding, "binding");
this.stores = Objects.requireNonNull(stores, "stores"); this.stores = Objects.requireNonNull(stores, "stores");
this.caveFluidPolicies = caveFluidPolicies; this.caveFluidPolicies = caveFluidPolicies;
@@ -153,7 +161,7 @@ public final class HydrologyRuntime {
HydrologyPlanCache cache = HydrologyPlanCache.create(settings, planners.regions(), planners.windows(), MultiBurst.hydrology, mainThread); HydrologyPlanCache cache = HydrologyPlanCache.create(settings, planners.regions(), planners.windows(), MultiBurst.hydrology, mainThread);
String dimensionKey = dimension.getLoadKey(); String dimensionKey = dimension.getLoadKey();
IrisWorld world = engine.getWorld(); IrisWorld world = engine.getWorld();
return new HydrologyRuntime(settings, cache, source::naturalHeight, mainThread, System::nanoTime, binding, return new HydrologyRuntime(settings, cache, source::naturalSurface, source, mainThread, System::nanoTime, binding,
identity -> store(world, dimensionKey, settings, identity), caveFluidPolicies); identity -> store(world, dimensionKey, settings, identity), caveFluidPolicies);
} }
@@ -234,7 +242,7 @@ public final class HydrologyRuntime {
requireOpen(); requireOpen();
long key = key(chunkX, chunkZ); long key = key(chunkX, chunkZ);
ChunkWaterRaster known = remembered(key, slot); ChunkWaterRaster known = remembered(key, slot);
return known != null ? known : remember(key, planned(chunkX, chunkZ, key)); return known != null ? known : build(chunkX, chunkZ, key, slot);
} }
public ChunkWaterRaster rasterSnapshot(int chunkX, int chunkZ) { public ChunkWaterRaster rasterSnapshot(int chunkX, int chunkZ) {
@@ -354,8 +362,28 @@ public final class HydrologyRuntime {
} }
} }
void boundPregeneration(Object epoch, int minRegionX, int minRegionZ, int maxRegionX, int maxRegionZ) {
Objects.requireNonNull(epoch, "epoch");
if (maxRegionX < minRegionX || maxRegionZ < minRegionZ) {
throw new IllegalArgumentException("Invalid pregeneration bounds " + minRegionX + "," + minRegionZ + " to " + maxRegionX + ","
+ maxRegionZ);
}
if (cache == null) {
return;
}
synchronized (prefetches) {
pregenAreas.put(epoch, new PregenArea(minRegionX - 1, minRegionZ - 1, maxRegionX + 1, maxRegionZ + 1));
prefetches.clear();
}
}
void cancelPregeneration(Object epoch) { void cancelPregeneration(Object epoch) {
if (cache != null) { if (cache != null) {
synchronized (prefetches) {
if (pregenAreas.remove(epoch) != null) {
prefetches.clear();
}
}
cache.cancel(epoch); cache.cancel(epoch);
} }
} }
@@ -501,6 +529,41 @@ public final class HydrologyRuntime {
} }
} }
private ChunkWaterRaster build(int chunkX, int chunkZ, long key, LastRaster slot) {
CompletableFuture<ChunkWaterRaster> claim = new CompletableFuture<>();
CompletableFuture<ChunkWaterRaster> running = building.putIfAbsent(key, claim);
if (running != null) {
ChunkWaterRaster built = built(running);
slot.remember(key, built);
return built;
}
try {
ChunkWaterRaster known = remembered(key, slot);
ChunkWaterRaster raster = known != null ? known : remember(key, planned(chunkX, chunkZ, key));
claim.complete(raster);
return raster;
} catch (RuntimeException | Error failure) {
claim.completeExceptionally(failure);
throw failure;
} finally {
building.remove(key, claim);
}
}
private static ChunkWaterRaster built(CompletableFuture<ChunkWaterRaster> running) {
try {
return running.join();
} catch (CompletionException failure) {
if (failure.getCause() instanceof RuntimeException cause) {
throw cause;
}
if (failure.getCause() instanceof Error cause) {
throw cause;
}
throw failure;
}
}
private ChunkWaterRaster planned(int chunkX, int chunkZ, long chunk) { private ChunkWaterRaster planned(int chunkX, int chunkZ, long chunk) {
try { try {
List<WindowKey> windowKeys = neighbourhood(chunkX, chunkZ); List<WindowKey> windowKeys = neighbourhood(chunkX, chunkZ);
@@ -667,7 +730,7 @@ public final class HydrologyRuntime {
&& (!known.prepared().isDone() || known.evictions() == evictions)) { && (!known.prepared().isDone() || known.evictions() == evictions)) {
return; return;
} }
CompletableFuture<Void> prepared = cache.prepareWindowsAround(chunkX, chunkZ, chunkX, chunkZ, 1 + settings.prefetchRings()); CompletableFuture<Void> prepared = prepareAround(chunkX, chunkZ, owner, 1 + settings.prefetchRings());
prefetches.put(owner, new WindowPrefetch(prepared, evictions)); prefetches.put(owner, new WindowPrefetch(prepared, evictions));
if (prefetches.size() > PREFETCH_CAPACITY) { if (prefetches.size() > PREFETCH_CAPACITY) {
prefetches.pollFirstEntry(); prefetches.pollFirstEntry();
@@ -675,6 +738,24 @@ public final class HydrologyRuntime {
} }
} }
private CompletableFuture<Void> prepareAround(int chunkX, int chunkZ, WindowKey owner, int rings) {
PregenArea covering = null;
for (PregenArea area : pregenAreas.values()) {
if (area.contains(owner)) {
covering = covering == null ? area : covering.union(area);
}
}
if (covering == null) {
return cache.prepareWindowsAround(chunkX, chunkZ, chunkX, chunkZ, rings);
}
int minX = Math.max(owner.wx() - rings, covering.minX());
int minZ = Math.max(owner.wz() - rings, covering.minZ());
int maxX = Math.min(owner.wx() + rings, covering.maxX());
int maxZ = Math.min(owner.wz() + rings, covering.maxZ());
return cache.prepareWindowsAround(minX * CHUNKS_PER_WINDOW, minZ * CHUNKS_PER_WINDOW, (maxX + 1) * CHUNKS_PER_WINDOW - 1,
(maxZ + 1) * CHUNKS_PER_WINDOW - 1, 0);
}
private static List<WindowKey> neighbourhood(int chunkX, int chunkZ) { private static List<WindowKey> neighbourhood(int chunkX, int chunkZ) {
WindowKey own = WindowKey.containing(chunkX << 4, chunkZ << 4); WindowKey own = WindowKey.containing(chunkX << 4, chunkZ << 4);
List<WindowKey> keys = new ArrayList<>(9); List<WindowKey> keys = new ArrayList<>(9);
@@ -754,6 +835,16 @@ public final class HydrologyRuntime {
private record WindowPrefetch(CompletableFuture<Void> prepared, long evictions) { private record WindowPrefetch(CompletableFuture<Void> prepared, long evictions) {
} }
private record PregenArea(int minX, int minZ, int maxX, int maxZ) {
private boolean contains(WindowKey key) {
return key.wx() > minX && key.wx() < maxX && key.wz() > minZ && key.wz() < maxZ;
}
private PregenArea union(PregenArea other) {
return new PregenArea(Math.min(minX, other.minX), Math.min(minZ, other.minZ), Math.max(maxX, other.maxX), Math.max(maxZ, other.maxZ));
}
}
private static final class LastRaster { private static final class LastRaster {
private long key; private long key;
private ChunkWaterRaster raster; private ChunkWaterRaster raster;
@@ -7,11 +7,13 @@ import art.arcane.iris.generation.hydrology.config.IrisHydrologyPolicy;
import art.arcane.iris.generation.hydrology.grid.HydrologySampleBuffer; import art.arcane.iris.generation.hydrology.grid.HydrologySampleBuffer;
import art.arcane.iris.generation.hydrology.grid.HydrologyTerrainSource; import art.arcane.iris.generation.hydrology.grid.HydrologyTerrainSource;
import art.arcane.iris.generation.hydrology.plan.HydrologySettings; import art.arcane.iris.generation.hydrology.plan.HydrologySettings;
import art.arcane.iris.generation.hydrology.raster.ChannelFluids;
import art.arcane.iris.generation.runtime.IrisComplex; import art.arcane.iris.generation.runtime.IrisComplex;
import art.arcane.iris.generation.terrain.InferredType; import art.arcane.iris.generation.terrain.InferredType;
import art.arcane.iris.generation.terrain.IrisDimension; import art.arcane.iris.generation.terrain.IrisDimension;
import art.arcane.iris.generation.terrain.IrisRegion; import art.arcane.iris.generation.terrain.IrisRegion;
import art.arcane.iris.studio.generation.BiomeBuffetLayout; import art.arcane.iris.studio.generation.BiomeBuffetLayout;
import art.arcane.volmlib.util.noise.CNG;
import art.arcane.volmlib.util.stream.ProceduralStream; import art.arcane.volmlib.util.stream.ProceduralStream;
import java.util.ArrayList; import java.util.ArrayList;
@@ -25,9 +27,10 @@ import java.util.concurrent.RejectedExecutionException;
import java.util.concurrent.atomic.AtomicBoolean; import java.util.concurrent.atomic.AtomicBoolean;
import java.util.function.Function; import java.util.function.Function;
public final class IrisHydrologyTerrainSource implements HydrologyTerrainSource { public final class IrisHydrologyTerrainSource implements HydrologyTerrainSource, ChannelFluids {
private static final int MINIMUM_SLICE = 256; private static final int MINIMUM_SLICE = 256;
private static final int POLICY_CAPACITY = 4096; private static final int POLICY_CAPACITY = 4096;
private static final long SALT_CLIMATE = 0x434c494d415445L;
private final IrisComplex complex; private final IrisComplex complex;
private final IrisDimension dimension; private final IrisDimension dimension;
@@ -40,6 +43,8 @@ public final class IrisHydrologyTerrainSource implements HydrologyTerrainSource
private final BiomeBuffetLayout buffet; private final BiomeBuffetLayout buffet;
private final IrisBiome focus; private final IrisBiome focus;
private final HydrologyBinding binding; private final HydrologyBinding binding;
private final HydrologySettings.Climate climate;
private final CNG wetness;
private final ConcurrentHashMap<PolicyKey, Policy> policies = new ConcurrentHashMap<>(); private final ConcurrentHashMap<PolicyKey, Policy> policies = new ConcurrentHashMap<>();
public IrisHydrologyTerrainSource(IrisComplex complex, IrisDimension dimension, HydrologySettings settings, ExecutorService pool, public IrisHydrologyTerrainSource(IrisComplex complex, IrisDimension dimension, HydrologySettings settings, ExecutorService pool,
@@ -55,6 +60,8 @@ public final class IrisHydrologyTerrainSource implements HydrologyTerrainSource
this.buffet = complex.getBiomeBuffet(); this.buffet = complex.getBiomeBuffet();
this.focus = complex.getFocusBiome(); this.focus = complex.getFocusBiome();
this.binding = Objects.requireNonNull(binding, "binding"); this.binding = Objects.requireNonNull(binding, "binding");
this.climate = settings.climate();
this.wetness = climate.enabled() ? climate.style().create(settings.seed() ^ SALT_CLIMATE, complex.getData()) : null;
} }
@Override @Override
@@ -73,8 +80,13 @@ public final class IrisHydrologyTerrainSource implements HydrologyTerrainSource
} }
} }
public int naturalHeight(int x, int z) { public double naturalSurface(int x, int z) {
return complex.naturalTrueHeight(x, z); return complex.naturalHeight(x, z);
}
@Override
public String channelFluid(int x, int z) {
return policy(biomes.get(x, z), regions.get(x, z)).fluid();
} }
@Override @Override
@@ -150,12 +162,17 @@ public final class IrisHydrologyTerrainSource implements HydrologyTerrainSource
Policy policy = policy(biomes.get(x, z), regions.get(x, z)); Policy policy = policy(biomes.get(x, z), regions.get(x, z));
out.height()[index] = (float) height; out.height()[index] = (float) height;
out.ocean()[index] = StrictMath.round(height) < settings.seaLevel() && oceanIntent(x, z); out.ocean()[index] = StrictMath.round(height) < settings.seaLevel() && oceanIntent(x, z);
out.rainfall()[index] = policy.rainfall(); float wet = policy.climateZones() ? wetness(x, z) : 1f;
out.pondDensity()[index] = policy.pondDensity(); out.rainfall()[index] = policy.rainfall() * wet;
out.pondDensity()[index] = policy.pondDensity() * wet;
out.permission()[index] = policy.permission(); out.permission()[index] = policy.permission();
out.fluidProfile()[index] = policy.fluid(); out.fluidProfile()[index] = policy.fluid();
} }
private float wetness(int x, int z) {
return wetness == null ? 1f : (float) climate.wetness(wetness.noise(x, z));
}
private boolean oceanIntent(int x, int z) { private boolean oceanIntent(int x, int z) {
BiomeBuffetLayout.Cell cell = buffet == null ? null : buffet.terrain(x, z); BiomeBuffetLayout.Cell cell = buffet == null ? null : buffet.terrain(x, z);
IrisBiome focused = cell == null ? focus : cell.biome(); IrisBiome focused = cell == null ? focus : cell.biome();
@@ -183,13 +200,14 @@ public final class IrisHydrologyTerrainSource implements HydrologyTerrainSource
IrisHydrologyPermission streams = inherited(biome, region, IrisHydrologyPolicy::getStreams, IrisHydrologyPermission.ALLOW); IrisHydrologyPermission streams = inherited(biome, region, IrisHydrologyPolicy::getStreams, IrisHydrologyPermission.ALLOW);
IrisHydrologyPermission ponds = inherited(biome, region, IrisHydrologyPolicy::getPonds, IrisHydrologyPermission.ALLOW); IrisHydrologyPermission ponds = inherited(biome, region, IrisHydrologyPolicy::getPonds, IrisHydrologyPermission.ALLOW);
String fluid = inherited(biome, region, IrisHydrologyPolicy::getFluid, null); String fluid = inherited(biome, region, IrisHydrologyPolicy::getFluid, null);
boolean climateZones = inherited(biome, region, IrisHydrologyPolicy::getClimateZones, Boolean.TRUE);
requireMultiplier("rainfall", rainfall, biome, region); requireMultiplier("rainfall", rainfall, biome, region);
requireMultiplier("pondDensity", pondDensity, biome, region); requireMultiplier("pondDensity", pondDensity, biome, region);
if (fluid != null && !settings.fluids().containsKey(fluid)) { if (fluid != null && !settings.fluids().containsKey(fluid)) {
throw new IllegalArgumentException("Hydrology policy for " + where(biome, region) + " names fluid '" + fluid throw new IllegalArgumentException("Hydrology policy for " + where(biome, region) + " names fluid '" + fluid
+ "', which is not one of the hydrology fluids " + settings.fluids().keySet()); + "', which is not one of the hydrology fluids " + settings.fluids().keySet());
} }
return new Policy((float) rainfall, (float) pondDensity, permission(streams, ponds), fluid); return new Policy((float) rainfall, (float) pondDensity, permission(streams, ponds), fluid, climateZones);
} }
private <T> T inherited(IrisBiome biome, IrisRegion region, Function<IrisHydrologyPolicy, T> field, T fallback) { private <T> T inherited(IrisBiome biome, IrisRegion region, Function<IrisHydrologyPolicy, T> field, T fallback) {
@@ -214,7 +232,7 @@ public final class IrisHydrologyTerrainSource implements HydrologyTerrainSource
return "biome " + (biome == null ? "none" : biome.getLoadKey()) + " in region " + (region == null ? "none" : region.getLoadKey()); return "biome " + (biome == null ? "none" : biome.getLoadKey()) + " in region " + (region == null ? "none" : region.getLoadKey());
} }
private record Policy(float rainfall, float pondDensity, byte permission, String fluid) { private record Policy(float rainfall, float pondDensity, byte permission, String fluid, boolean climateZones) {
} }
private record PolicyKey(IrisBiome biome, IrisRegion region) { private record PolicyKey(IrisBiome biome, IrisRegion region) {
@@ -1,34 +1,59 @@
package art.arcane.iris.generation.hydrology.grid; package art.arcane.iris.generation.hydrology.grid;
import java.util.ArrayList;
import java.util.Arrays; import java.util.Arrays;
import java.util.Comparator;
import java.util.List;
import java.util.Map;
import java.util.Objects; import java.util.Objects;
import java.util.concurrent.CancellationException; import java.util.concurrent.CancellationException;
import java.util.concurrent.CompletableFuture; import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.ExecutionException; import java.util.concurrent.ExecutionException;
import java.util.concurrent.atomic.AtomicBoolean; import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.atomic.AtomicReferenceArray;
public final class HeightLattice { public final class HeightLattice {
private static final int TILE_SHIFT = 5;
private static final int COLUMN_TILE_SHIFT = 3;
private static final int COLUMN_CELLS = 1 << 20;
private static final CompletableFuture<Void> READY = CompletableFuture.completedFuture(null);
private static final int[] NO_NODES = new int[0];
private static final Tile[] NO_TILES = new Tile[0];
private final HydrologyTerrainSource source; private final HydrologyTerrainSource source;
private final int cell; private final int cell;
private final boolean baseOnly; private final boolean baseOnly;
private final int capacityCells; private final int tileShift;
private final ConcurrentHashMap<Long, Slot> cells; private final int tileMask;
private final ConcurrentLinkedQueue<Long> insertionOrder; private final int tileCells;
private final int capacityTiles;
private final ConcurrentHashMap<Long, Tile> tiles;
private final AtomicLong clock;
private final AtomicBoolean evicting; private final AtomicBoolean evicting;
private final HeightLattice columns;
public HeightLattice(HydrologyTerrainSource source, int cell, boolean baseOnly, int capacityCells) { public HeightLattice(HydrologyTerrainSource source, int cell, boolean baseOnly, int capacityCells) {
this(source, cell, baseOnly, capacityCells, TILE_SHIFT, new HeightLattice(source, 1, baseOnly, COLUMN_CELLS, COLUMN_TILE_SHIFT, null));
}
private HeightLattice(HydrologyTerrainSource source, int cell, boolean baseOnly, int capacityCells, int tileShift, HeightLattice columns) {
if (cell <= 0 || capacityCells <= 0) { if (cell <= 0 || capacityCells <= 0) {
throw new IllegalArgumentException("Invalid lattice cell=" + cell + " capacityCells=" + capacityCells); throw new IllegalArgumentException("Invalid lattice cell=" + cell + " capacityCells=" + capacityCells);
} }
this.source = Objects.requireNonNull(source, "source"); this.source = Objects.requireNonNull(source, "source");
this.cell = cell; this.cell = cell;
this.baseOnly = baseOnly; this.baseOnly = baseOnly;
this.capacityCells = capacityCells; this.tileShift = tileShift;
this.cells = new ConcurrentHashMap<>(); this.tileMask = (1 << tileShift) - 1;
this.insertionOrder = new ConcurrentLinkedQueue<>(); this.tileCells = 1 << (2 * tileShift);
this.capacityTiles = Math.ceilDiv(capacityCells, tileCells);
this.tiles = new ConcurrentHashMap<>();
this.clock = new AtomicLong();
this.evicting = new AtomicBoolean(); this.evicting = new AtomicBoolean();
this.columns = columns;
} }
public int cell() { public int cell() {
@@ -67,19 +92,24 @@ public final class HeightLattice {
throw new IllegalArgumentException("Column sample has " + xs.length + " x and " + zs.length + " z coordinates"); throw new IllegalArgumentException("Column sample has " + xs.length + " x and " + zs.length + " z coordinates");
} }
requireCapacity(out, xs.length); requireCapacity(out, xs.length);
if (xs.length > 0) { if (xs.length == 0) {
source.sample(xs, zs, out, baseOnly); return;
} }
if (columns == null) {
source.sample(xs, zs, out, baseOnly);
return;
}
columns.resolve(new Scatter(xs, zs), xs.length, out);
} }
private void resolve(Nodes nodes, int count, HydrologySampleBuffer out) { private void resolve(Nodes nodes, int count, HydrologySampleBuffer out) {
Slot[] slots = new Slot[count]; Pass pass = new Pass(nodes, out, clock.incrementAndGet());
int unresolved = count; int[] open = null;
while (unresolved > 0) { int remaining = count;
CompletableFuture<HydrologySampleBuffer> batch = claimAndSample(nodes, slots); while (remaining > 0) {
unresolved = awaitForeign(batch, slots); open = pass.run(open, remaining);
remaining = open == null ? 0 : open.length;
} }
copy(slots, out);
evictOverflow(); evictOverflow();
} }
@@ -94,130 +124,33 @@ public final class HeightLattice {
Math.multiplyExact(Math.addExact(cell0, Math.multiplyExact(extent - 1, stride)), cell); Math.multiplyExact(Math.addExact(cell0, Math.multiplyExact(extent - 1, stride)), cell);
} }
private CompletableFuture<HydrologySampleBuffer> claimAndSample(Nodes nodes, Slot[] slots) { private Tile tile(int tileX, int tileZ) {
CompletableFuture<HydrologySampleBuffer> batch = new CompletableFuture<>(); Long key = key(tileX, tileZ);
try { Tile known = tiles.get(key);
int[] xs = new int[slots.length]; if (known != null) {
int[] zs = new int[slots.length]; return known;
int claimed = claim(batch, nodes, slots, xs, zs);
if (claimed > 0) {
HydrologySampleBuffer samples = new HydrologySampleBuffer(claimed);
source.sample(Arrays.copyOf(xs, claimed), Arrays.copyOf(zs, claimed), samples, baseOnly);
batch.complete(samples);
}
} catch (Throwable failure) {
try {
release(batch, nodes, slots);
} finally {
batch.completeExceptionally(failure);
}
throw failure;
}
return batch;
}
private int claim(CompletableFuture<HydrologySampleBuffer> batch, Nodes nodes, Slot[] slots, int[] xs, int[] zs) {
int claimed = 0;
for (int index = 0; index < slots.length; index++) {
if (slots[index] != null) {
continue;
}
int cellX = nodes.cellX(index);
int cellZ = nodes.cellZ(index);
Long key = key(cellX, cellZ);
Slot existing = cells.get(key);
if (existing == null) {
Slot fresh = new Slot(batch, claimed);
existing = cells.putIfAbsent(key, fresh);
if (existing == null) {
slots[index] = fresh;
insertionOrder.offer(key);
xs[claimed] = cellX * cell;
zs[claimed] = cellZ * cell;
claimed++;
continue;
}
}
slots[index] = existing;
}
return claimed;
}
private void release(CompletableFuture<HydrologySampleBuffer> batch, Nodes nodes, Slot[] slots) {
for (int index = 0; index < slots.length; index++) {
Slot slot = slots[index];
if (slot != null && slot.batch() == batch) {
cells.remove(key(nodes.cellX(index), nodes.cellZ(index)), slot);
}
}
}
private static int awaitForeign(CompletableFuture<HydrologySampleBuffer> own, Slot[] slots) {
int unresolved = 0;
for (int index = 0; index < slots.length; index++) {
CompletableFuture<HydrologySampleBuffer> batch = slots[index].batch();
if (batch != own && !await(batch)) {
slots[index] = null;
unresolved++;
}
}
return unresolved;
}
private static boolean await(CompletableFuture<HydrologySampleBuffer> batch) {
if (!batch.isDone()) {
try {
batch.get();
} catch (ExecutionException | CancellationException failure) {
return false;
} catch (InterruptedException interruption) {
Thread.currentThread().interrupt();
throw new CancellationException("Hydrology lattice wait interrupted.");
}
}
return !batch.isCompletedExceptionally();
}
private static void copy(Slot[] slots, HydrologySampleBuffer out) {
float[] height = out.height();
boolean[] ocean = out.ocean();
float[] rainfall = out.rainfall();
float[] pondDensity = out.pondDensity();
byte[] permission = out.permission();
String[] fluidProfile = out.fluidProfile();
for (int index = 0; index < slots.length; index++) {
Slot slot = slots[index];
HydrologySampleBuffer samples = slot.batch().join();
int sample = slot.index();
height[index] = samples.height()[sample];
ocean[index] = samples.ocean()[sample];
rainfall[index] = samples.rainfall()[sample];
pondDensity[index] = samples.pondDensity()[sample];
permission[index] = samples.permission()[sample];
fluidProfile[index] = samples.fluidProfile()[sample];
} }
return tiles.computeIfAbsent(key, ignored -> new Tile(tileCells));
} }
private void evictOverflow() { private void evictOverflow() {
if (cells.size() <= capacityCells || !evicting.compareAndSet(false, true)) { if (tiles.size() <= capacityTiles || !evicting.compareAndSet(false, true)) {
return; return;
} }
try { try {
int excess = cells.size() - capacityCells; List<IdleTile> idle = new ArrayList<>(tiles.size());
while (excess > 0) { for (Map.Entry<Long, Tile> entry : tiles.entrySet()) {
Long key = insertionOrder.poll(); Tile tile = entry.getValue();
if (key == null) { if (tile.pending.get() == 0) {
return; idle.add(new IdleTile(entry.getKey(), tile, tile.used));
} }
Slot slot = cells.get(key); }
if (slot == null) { idle.sort(Comparator.comparingLong(IdleTile::used));
continue; int target = capacityTiles - (capacityTiles >> 3);
} for (int index = 0; index < idle.size() && tiles.size() > target; index++) {
if (!slot.batch().isDone()) { IdleTile candidate = idle.get(index);
insertionOrder.offer(key); if (candidate.tile().pending.get() == 0) {
excess--; tiles.remove(candidate.key(), candidate.tile());
} else if (cells.remove(key, slot)) {
excess--;
} }
} }
} finally { } finally {
@@ -232,7 +165,219 @@ public final class HeightLattice {
return mixed ^ (mixed >>> 31); return mixed ^ (mixed >>> 31);
} }
private record Slot(CompletableFuture<HydrologySampleBuffer> batch, int index) { private static boolean await(CompletableFuture<Void> batch) {
if (!batch.isDone()) {
try {
batch.get();
} catch (ExecutionException | CancellationException failure) {
return false;
} catch (InterruptedException interruption) {
Thread.currentThread().interrupt();
throw new CancellationException("Hydrology lattice wait interrupted.");
}
}
return !batch.isCompletedExceptionally();
}
private static void copy(HydrologySampleBuffer samples, int sample, HydrologySampleBuffer out, int index) {
out.height()[index] = samples.height()[sample];
out.ocean()[index] = samples.ocean()[sample];
out.rainfall()[index] = samples.rainfall()[sample];
out.pondDensity()[index] = samples.pondDensity()[sample];
out.permission()[index] = samples.permission()[sample];
out.fluidProfile()[index] = samples.fluidProfile()[sample];
}
private final class Pass {
private final Nodes nodes;
private final HydrologySampleBuffer out;
private final long stamp;
private int[] claimedIndex = NO_NODES;
private Tile[] claimedTile = NO_TILES;
private int[] claimedLocal = NO_NODES;
private int claimed;
private int[] waitingIndex = NO_NODES;
private Tile[] waitingTile = NO_TILES;
private int[] waitingLocal = NO_NODES;
private int waiting;
private Tile current;
private int currentX;
private int currentZ;
private Pass(Nodes nodes, HydrologySampleBuffer out, long stamp) {
this.nodes = nodes;
this.out = out;
this.stamp = stamp;
}
private int[] run(int[] open, int remaining) {
claimed = 0;
waiting = 0;
CompletableFuture<Void> batch = new CompletableFuture<>();
for (int position = 0; position < remaining; position++) {
visit(open == null ? position : open[position], batch);
}
if (claimed > 0) {
sample(batch);
}
return awaitWaiting();
}
private void visit(int index, CompletableFuture<Void> batch) {
int cellX = nodes.cellX(index);
int cellZ = nodes.cellZ(index);
Tile tile = tileOf(cellX >> tileShift, cellZ >> tileShift);
int local = ((cellZ & tileMask) << tileShift) | (cellX & tileMask);
while (true) {
CompletableFuture<Void> mark = tile.marks.get(local);
if (mark == READY) {
tile.copy(local, out, index);
return;
}
if (mark != null) {
addWaiting(index, tile, local);
return;
}
if (tile.marks.compareAndSet(local, null, batch)) {
tile.pending.incrementAndGet();
addClaimed(index, tile, local);
return;
}
}
}
private Tile tileOf(int tileX, int tileZ) {
if (current == null || currentX != tileX || currentZ != tileZ) {
current = tile(tileX, tileZ);
currentX = tileX;
currentZ = tileZ;
current.used = stamp;
}
return current;
}
private void sample(CompletableFuture<Void> batch) {
try {
int[] xs = new int[claimed];
int[] zs = new int[claimed];
for (int position = 0; position < claimed; position++) {
xs[position] = nodes.cellX(claimedIndex[position]) * cell;
zs[position] = nodes.cellZ(claimedIndex[position]) * cell;
}
HydrologySampleBuffer samples = new HydrologySampleBuffer(claimed);
source.sample(xs, zs, samples, baseOnly);
for (int position = 0; position < claimed; position++) {
Tile tile = claimedTile[position];
int local = claimedLocal[position];
tile.store(local, samples, position);
tile.marks.set(local, READY);
tile.pending.decrementAndGet();
copy(samples, position, out, claimedIndex[position]);
}
batch.complete(null);
} catch (Throwable failure) {
try {
release(batch);
} finally {
batch.completeExceptionally(failure);
}
throw failure;
}
}
private void release(CompletableFuture<Void> batch) {
for (int position = 0; position < claimed; position++) {
Tile tile = claimedTile[position];
if (tile.marks.compareAndSet(claimedLocal[position], batch, null)) {
tile.pending.decrementAndGet();
}
}
}
private int[] awaitWaiting() {
int[] open = null;
int unresolved = 0;
for (int position = 0; position < waiting; position++) {
Tile tile = waitingTile[position];
int local = waitingLocal[position];
CompletableFuture<Void> mark = tile.marks.get(local);
if (mark != READY && (mark == null || !await(mark) || tile.marks.get(local) != READY)) {
if (open == null) {
open = new int[waiting - position];
}
open[unresolved++] = waitingIndex[position];
continue;
}
tile.copy(local, out, waitingIndex[position]);
}
return open == null ? null : Arrays.copyOf(open, unresolved);
}
private void addClaimed(int index, Tile tile, int local) {
if (claimed == claimedIndex.length) {
int grown = Math.max(16, claimed * 2);
claimedIndex = Arrays.copyOf(claimedIndex, grown);
claimedTile = Arrays.copyOf(claimedTile, grown);
claimedLocal = Arrays.copyOf(claimedLocal, grown);
}
claimedIndex[claimed] = index;
claimedTile[claimed] = tile;
claimedLocal[claimed] = local;
claimed++;
}
private void addWaiting(int index, Tile tile, int local) {
if (waiting == waitingIndex.length) {
int grown = Math.max(16, waiting * 2);
waitingIndex = Arrays.copyOf(waitingIndex, grown);
waitingTile = Arrays.copyOf(waitingTile, grown);
waitingLocal = Arrays.copyOf(waitingLocal, grown);
}
waitingIndex[waiting] = index;
waitingTile[waiting] = tile;
waitingLocal[waiting] = local;
waiting++;
}
}
private static final class Tile {
private final float[] height;
private final boolean[] ocean;
private final float[] rainfall;
private final float[] pondDensity;
private final byte[] permission;
private final String[] fluidProfile;
private final AtomicReferenceArray<CompletableFuture<Void>> marks;
private final AtomicInteger pending = new AtomicInteger();
private volatile long used;
private Tile(int cells) {
this.height = new float[cells];
this.ocean = new boolean[cells];
this.rainfall = new float[cells];
this.pondDensity = new float[cells];
this.permission = new byte[cells];
this.fluidProfile = new String[cells];
this.marks = new AtomicReferenceArray<>(cells);
}
private void store(int local, HydrologySampleBuffer samples, int sample) {
height[local] = samples.height()[sample];
ocean[local] = samples.ocean()[sample];
rainfall[local] = samples.rainfall()[sample];
pondDensity[local] = samples.pondDensity()[sample];
permission[local] = samples.permission()[sample];
fluidProfile[local] = samples.fluidProfile()[sample];
}
private void copy(int local, HydrologySampleBuffer out, int index) {
out.height()[index] = height[local];
out.ocean()[index] = ocean[local];
out.rainfall()[index] = rainfall[local];
out.pondDensity()[index] = pondDensity[local];
out.permission()[index] = permission[local];
out.fluidProfile()[index] = fluidProfile[local];
}
} }
private interface Nodes { private interface Nodes {
@@ -241,6 +386,9 @@ public final class HeightLattice {
int cellZ(int index); int cellZ(int index);
} }
private record IdleTile(Long key, Tile tile, long used) {
}
private record Span(int cellX0, int cellZ0, int width, int stride) implements Nodes { private record Span(int cellX0, int cellZ0, int width, int stride) implements Nodes {
@Override @Override
public int cellX(int index) { public int cellX(int index) {
@@ -32,10 +32,10 @@ import java.util.function.Consumer;
import java.util.function.Function; import java.util.function.Function;
public final class HydrologyPlanCache { public final class HydrologyPlanCache {
private static final int FRONT = 3; private static final int FINE_FRONT = 9;
public static final int WINDOW_LATTICE_CELLS = frontCells(WindowKey.SIZE, WindowPlanner.CELL); private static final int REGION_FRONT = 5;
public static final int FINE_LATTICE_CELLS = frontCells(WindowKey.SIZE, WindowPlanner.FINE_CELL); public static final int FINE_LATTICE_CELLS = frontCells(FINE_FRONT, WindowKey.SIZE, WindowPlanner.FINE_CELL);
public static final int REGION_LATTICE_CELLS = frontCells(RegionKey.SIZE, RegionPlanner.CELL); public static final int REGION_LATTICE_CELLS = frontCells(REGION_FRONT, RegionKey.SIZE, RegionPlanner.CELL);
private static final int WINDOW_CAPACITY = 128; private static final int WINDOW_CAPACITY = 128;
private static final int REGION_CAPACITY = 64; private static final int REGION_CAPACITY = 64;
private static final long DEMAND = Long.MIN_VALUE; private static final long DEMAND = Long.MIN_VALUE;
@@ -312,8 +312,8 @@ public final class HydrologyPlanCache {
return Math.max(1, parallelism / 2); return Math.max(1, parallelism / 2);
} }
private static int frontCells(int unit, int cell) { private static int frontCells(int front, int unit, int cell) {
int cells = (FRONT + 2) * unit / cell; int cells = (front + 2) * unit / cell;
return cells * cells; return cells * cells;
} }
@@ -5,6 +5,7 @@ import art.arcane.iris.generation.decoration.IrisDecorationPart;
import art.arcane.iris.generation.decoration.IrisDecorator; import art.arcane.iris.generation.decoration.IrisDecorator;
import art.arcane.iris.generation.hydrology.config.IrisCavePools; import art.arcane.iris.generation.hydrology.config.IrisCavePools;
import art.arcane.iris.generation.hydrology.config.IrisFluidProfile; import art.arcane.iris.generation.hydrology.config.IrisFluidProfile;
import art.arcane.iris.generation.hydrology.config.IrisHydrologyClimate;
import art.arcane.iris.generation.hydrology.config.IrisHydrology; import art.arcane.iris.generation.hydrology.config.IrisHydrology;
import art.arcane.iris.generation.hydrology.config.IrisHydrologyDeformation; import art.arcane.iris.generation.hydrology.config.IrisHydrologyDeformation;
import art.arcane.iris.generation.hydrology.config.IrisHydrologyDrainage; import art.arcane.iris.generation.hydrology.config.IrisHydrologyDrainage;
@@ -20,6 +21,7 @@ import art.arcane.iris.generation.hydrology.config.IrisPondShore;
import art.arcane.iris.generation.hydrology.config.IrisSeaInlets; import art.arcane.iris.generation.hydrology.config.IrisSeaInlets;
import art.arcane.iris.generation.hydrology.underground.SeaInletRasterizer; import art.arcane.iris.generation.hydrology.underground.SeaInletRasterizer;
import art.arcane.iris.generation.hydrology.water.HydrologyTier; import art.arcane.iris.generation.hydrology.water.HydrologyTier;
import art.arcane.iris.generation.noise.IrisGeneratorStyle;
import art.arcane.iris.generation.terrain.IrisMaterialPalette; import art.arcane.iris.generation.terrain.IrisMaterialPalette;
import com.google.gson.Gson; import com.google.gson.Gson;
@@ -51,6 +53,7 @@ public final class HydrologySettings {
private final int maxY; private final int maxY;
private final boolean enabled; private final boolean enabled;
private final float density; private final float density;
private final Climate climate;
private final Drainage drainage; private final Drainage drainage;
private final List<HydrologyTier> tiers; private final List<HydrologyTier> tiers;
private final Ponds ponds; private final Ponds ponds;
@@ -72,6 +75,7 @@ public final class HydrologySettings {
throw new IllegalArgumentException("Hydrology density must be within 0..1, got " + config.getDensity()); throw new IllegalArgumentException("Hydrology density must be within 0..1, got " + config.getDensity());
} }
this.density = (float) config.getDensity(); this.density = (float) config.getDensity();
this.climate = climate(config.getClimate());
this.drainage = drainage(config.getDrainage()); this.drainage = drainage(config.getDrainage());
this.tiers = tiers(config.getTiers()); this.tiers = tiers(config.getTiers());
this.ponds = ponds(config.getPonds()); this.ponds = ponds(config.getPonds());
@@ -135,6 +139,10 @@ public final class HydrologySettings {
return density; return density;
} }
public Climate climate() {
return climate;
}
public Drainage drainage() { public Drainage drainage() {
return drainage; return drainage;
} }
@@ -167,6 +175,22 @@ public final class HydrologySettings {
return persist; return persist;
} }
private static Climate climate(IrisHydrologyClimate config) {
Objects.requireNonNull(config, "Hydrology climate is missing");
IrisGeneratorStyle style = Objects.requireNonNull(config.getStyle(), "Hydrology climate.style is missing");
float dryness = unit(config.getDryness(), "climate.dryness");
float transition = unit(config.getTransition(), "climate.transition");
float dryRainfall = unit(config.getDryRainfall(), "climate.dryRainfall");
return new Climate(config.isEnabled(), style, dryness, transition, dryRainfall);
}
private static float unit(double value, String field) {
if (!Double.isFinite(value) || value < 0D || value > 1D) {
throw new IllegalArgumentException("Hydrology " + field + " must be within 0..1, got " + value);
}
return (float) value;
}
private static Drainage drainage(IrisHydrologyDrainage config) { private static Drainage drainage(IrisHydrologyDrainage config) {
return new Drainage((float) config.getFillEpsilon(), config.getBreachDepth(), config.isWetnessThinning(), return new Drainage((float) config.getFillEpsilon(), config.getBreachDepth(), config.isWetnessThinning(),
(float) config.getMinimumHeadKeep(), (float) config.getHeadRelief(), (float) config.getSaturationCatchment()); (float) config.getMinimumHeadKeep(), (float) config.getHeadRelief(), (float) config.getSaturationCatchment());
@@ -468,6 +492,7 @@ public final class HydrologySettings {
private long computeFingerprint() { private long computeFingerprint() {
Fingerprint hash = new Fingerprint().add(seed).add(seaLevel).add(minY).add(maxY).add(enabled) Fingerprint hash = new Fingerprint().add(seed).add(seaLevel).add(minY).add(maxY).add(enabled)
.add(density).add(prefetchRings).add(persist); .add(density).add(prefetchRings).add(persist);
addClimate(hash);
addDrainage(hash); addDrainage(hash);
addTiers(hash); addTiers(hash);
addPonds(hash); addPonds(hash);
@@ -477,6 +502,12 @@ public final class HydrologySettings {
return hash.value(); return hash.value();
} }
private void addClimate(Fingerprint hash) {
if (climate.enabled()) {
hash.add(true).add(PALETTE_JSON.toJson(climate.style())).add(climate.dryness()).add(climate.transition()).add(climate.dryRainfall());
}
}
private void addDrainage(Fingerprint hash) { private void addDrainage(Fingerprint hash) {
hash.add(drainage.fillEpsilon()).add(drainage.breachDepth()).add(drainage.wetnessThinning()) hash.add(drainage.fillEpsilon()).add(drainage.breachDepth()).add(drainage.wetnessThinning())
.add(drainage.minimumHeadKeep()).add(drainage.headRelief()).add(drainage.saturationCatchment()); .add(drainage.minimumHeadKeep()).add(drainage.headRelief()).add(drainage.saturationCatchment());
@@ -535,6 +566,23 @@ public final class HydrologySettings {
public record IntRange(int min, int max) { public record IntRange(int min, int max) {
} }
public record Climate(boolean enabled, IrisGeneratorStyle style, float dryness, float transition, float dryRainfall) {
public double wetness(double noise) {
if (!enabled || dryness <= 0f) {
return 1D;
}
double half = 0.5D * transition;
double share;
if (half <= 0D) {
share = noise < dryness ? 0D : 1D;
} else {
double t = Math.clamp((noise - dryness + half) / transition, 0D, 1D);
share = t * t * (3D - 2D * t);
}
return dryRainfall + (1D - dryRainfall) * share;
}
}
public record Drainage(float fillEpsilon, int breachDepth, boolean wetnessThinning, float minimumHeadKeep, float headRelief, public record Drainage(float fillEpsilon, int breachDepth, boolean wetnessThinning, float minimumHeadKeep, float headRelief,
float saturationCatchment) { float saturationCatchment) {
} }
@@ -36,6 +36,7 @@ public final class WindowPlanner {
public static final int CELL = 8; public static final int CELL = 8;
public static final int FINE_CELL = 4; public static final int FINE_CELL = 4;
private static final int WINDOW_CELLS = WindowKey.SIZE / CELL; private static final int WINDOW_CELLS = WindowKey.SIZE / CELL;
private static final int FINE_STRIDE = CELL / FINE_CELL;
private static final int SPAN = 3 * WINDOW_CELLS; private static final int SPAN = 3 * WINDOW_CELLS;
private static final int REACH_OVERHANG = 256; private static final int REACH_OVERHANG = 256;
private static final float BASIN_DEPTH = 0.5f; private static final float BASIN_DEPTH = 0.5f;
@@ -43,22 +44,17 @@ public final class WindowPlanner {
private static final long SALT_HEAD = 0x5354524541484dL; private static final long SALT_HEAD = 0x5354524541484dL;
private final HydrologySettings settings; private final HydrologySettings settings;
private final HeightLattice lattice;
private final HeightLattice fine; private final HeightLattice fine;
private final PondPlanner ponds; private final PondPlanner ponds;
private final ReachShaper shaper; private final ReachShaper shaper;
private final SeaInletPlanner inlets; private final SeaInletPlanner inlets;
public WindowPlanner(HydrologySettings settings, HeightLattice lattice, HeightLattice fine, PondPlanner ponds, ReachShaper shaper) { public WindowPlanner(HydrologySettings settings, HeightLattice fine, PondPlanner ponds, ReachShaper shaper) {
this.settings = Objects.requireNonNull(settings, "settings"); this.settings = Objects.requireNonNull(settings, "settings");
this.lattice = Objects.requireNonNull(lattice, "lattice");
this.fine = Objects.requireNonNull(fine, "fine"); this.fine = Objects.requireNonNull(fine, "fine");
this.ponds = Objects.requireNonNull(ponds, "ponds"); this.ponds = Objects.requireNonNull(ponds, "ponds");
this.shaper = Objects.requireNonNull(shaper, "shaper"); this.shaper = Objects.requireNonNull(shaper, "shaper");
this.inlets = new SeaInletPlanner(settings); this.inlets = new SeaInletPlanner(settings);
if (lattice.cell() != CELL) {
throw new IllegalArgumentException("Window lattice cell " + lattice.cell() + " must be " + CELL);
}
if (fine.cell() != FINE_CELL) { if (fine.cell() != FINE_CELL) {
throw new IllegalArgumentException("Window fine lattice cell " + fine.cell() + " must be " + FINE_CELL); throw new IllegalArgumentException("Window fine lattice cell " + fine.cell() + " must be " + FINE_CELL);
} }
@@ -87,7 +83,7 @@ public final class WindowPlanner {
int cellX = Math.multiplyExact(key.wx() - 1, WINDOW_CELLS); int cellX = Math.multiplyExact(key.wx() - 1, WINDOW_CELLS);
int cellZ = Math.multiplyExact(key.wz() - 1, WINDOW_CELLS); int cellZ = Math.multiplyExact(key.wz() - 1, WINDOW_CELLS);
HydrologySampleBuffer samples = new HydrologySampleBuffer(SPAN * SPAN); HydrologySampleBuffer samples = new HydrologySampleBuffer(SPAN * SPAN);
lattice.fill(cellX, cellZ, SPAN, SPAN, samples); fine.fill(cellX * FINE_STRIDE, cellZ * FINE_STRIDE, SPAN, SPAN, FINE_STRIDE, samples);
DemGrid raw = new DemGrid(cellX * CELL, cellZ * CELL, CELL, SPAN, SPAN); DemGrid raw = new DemGrid(cellX * CELL, cellZ * CELL, CELL, SPAN, SPAN);
System.arraycopy(samples.height(), 0, raw.values(), 0, SPAN * SPAN); System.arraycopy(samples.height(), 0, raw.values(), 0, SPAN * SPAN);
List<Pond> lakes = regionLakes(plans.values()); List<Pond> lakes = regionLakes(plans.values());
@@ -6,6 +6,7 @@ final class BedProfile {
private static final long SALT_BROAD = 0x42454442524f4144L; private static final long SALT_BROAD = 0x42454442524f4144L;
private static final long SALT_DETAIL = 0x4245444445544149L; private static final long SALT_DETAIL = 0x4245444445544149L;
private static final long SALT_TEXTURE = 0x4245445445585455L; private static final long SALT_TEXTURE = 0x4245445445585455L;
private static final long SALT_ROUGHNESS = 0x42414e4b524f5547L;
private BedProfile() { private BedProfile() {
} }
@@ -18,6 +19,10 @@ final class BedProfile {
return 0.35D * noise(seed, SALT_TEXTURE, x, z, 2); return 0.35D * noise(seed, SALT_TEXTURE, x, z, 2);
} }
static double roughness(long seed, int x, int z) {
return noise(seed, SALT_ROUGHNESS, x, z, 4) - 0.5D;
}
static double position(double u, double variation) { static double position(double u, double variation) {
double bounded = Math.clamp(u, 0D, 1D); double bounded = Math.clamp(u, 0D, 1D);
return bounded + 0.8D * variation * bounded * (1D - bounded); return bounded + 0.8D * variation * bounded * (1D - bounded);
@@ -0,0 +1,8 @@
package art.arcane.iris.generation.hydrology.raster;
@FunctionalInterface
public interface ChannelFluids {
ChannelFluids TIER = (x, z) -> null;
String channelFluid(int x, int z);
}
@@ -12,6 +12,7 @@ import art.arcane.iris.generation.hydrology.water.PondMask;
import art.arcane.iris.generation.hydrology.water.Reach; import art.arcane.iris.generation.hydrology.water.Reach;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashMap; import java.util.HashMap;
import java.util.HashSet; import java.util.HashSet;
import java.util.List; import java.util.List;
@@ -28,17 +29,20 @@ final class ChunkFeatures {
private final List<Pond> ponds; private final List<Pond> ponds;
private final List<Pond> lakes; private final List<Pond> lakes;
private final List<SeaInlet> inlets; private final List<SeaInlet> inlets;
private final int[] inletBoxes;
private final int[] lakeBoxes; private final int[] lakeBoxes;
private final ReachSegments segments = new ReachSegments(SEGMENT_CAPACITY); private final ReachSegments segments = new ReachSegments(SEGMENT_CAPACITY);
private final MaskEdges pondShores = new MaskEdges(MaskEdges.Kind.WATER); private final MaskEdges pondShores = new MaskEdges(MaskEdges.Kind.WATER);
private final MaskEdges pondBanks = new MaskEdges(MaskEdges.Kind.BANK); private final MaskEdges pondBanks = new MaskEdges(MaskEdges.Kind.BANK);
private final MaskEdges lakeReach = new MaskEdges(MaskEdges.Kind.GROWN_WATER); private final MaskEdges lakeReach = new MaskEdges(MaskEdges.Kind.GROWN_WATER);
private boolean inert;
private ChunkFeatures(List<Reach> reaches, List<Pond> ponds, List<Pond> lakes, List<SeaInlet> inlets) { private ChunkFeatures(List<Reach> reaches, List<Pond> ponds, List<Pond> lakes, List<SeaInlet> inlets, int[] inletBoxes) {
this.reaches = reaches; this.reaches = reaches;
this.ponds = ponds; this.ponds = ponds;
this.lakes = lakes; this.lakes = lakes;
this.inlets = inlets; this.inlets = inlets;
this.inletBoxes = inletBoxes;
this.lakeBoxes = new int[4 * lakes.size()]; this.lakeBoxes = new int[4 * lakes.size()];
} }
@@ -48,12 +52,15 @@ final class ChunkFeatures {
List<Reach> reaches = new ArrayList<>(); List<Reach> reaches = new ArrayList<>();
List<Pond> ponds = new ArrayList<>(); List<Pond> ponds = new ArrayList<>();
List<SeaInlet> inlets = new ArrayList<>(); List<SeaInlet> inlets = new ArrayList<>();
int[] inletBoxes = new int[0];
for (IndexedWindow window : windows) { for (IndexedWindow window : windows) {
for (SeaInlet inlet : window.plan().inlets()) { for (SeaInlet inlet : window.plan().inlets()) {
int[] bounds = SeaInletRasterizer.bounds(inlet); int[] bounds = SeaInletRasterizer.bounds(inlet);
int padding = RiffleProfile.RADIUS + 1; int padding = RiffleProfile.RADIUS + 1;
if (bounds[0] <= chunkX * CHUNK + CHUNK - 1 + padding && bounds[2] >= chunkX * CHUNK - padding if (bounds[0] <= chunkX * CHUNK + CHUNK - 1 + padding && bounds[2] >= chunkX * CHUNK - padding
&& bounds[1] <= chunkZ * CHUNK + CHUNK - 1 + padding && bounds[3] >= chunkZ * CHUNK - padding) { && bounds[1] <= chunkZ * CHUNK + CHUNK - 1 + padding && bounds[3] >= chunkZ * CHUNK - padding) {
inletBoxes = Arrays.copyOf(inletBoxes, inletBoxes.length + 4);
System.arraycopy(bounds, 0, inletBoxes, 4 * inlets.size(), 4);
inlets.add(inlet); inlets.add(inlet);
} }
} }
@@ -64,17 +71,22 @@ final class ChunkFeatures {
} }
} }
List<Pond> lakes = lakes(chunkX, chunkZ, halo, lakeRegionPlans, lakeRegions); List<Pond> lakes = lakes(chunkX, chunkZ, halo, lakeRegionPlans, lakeRegions);
ChunkFeatures features = new ChunkFeatures(reaches, ponds, lakes, inlets); ChunkFeatures features = new ChunkFeatures(reaches, ponds, lakes, inlets, inletBoxes);
features.index(settings, chunkX, chunkZ); features.index(settings, chunkX, chunkZ);
return features; return features;
} }
boolean empty() { boolean inert() {
return reaches.isEmpty() && ponds.isEmpty() && lakes.isEmpty() && inlets.isEmpty(); return inert;
} }
SeaInlet openInletAt(int x, int z, int[] column) { SeaInlet openInletAt(int x, int z, int[] column) {
for (SeaInlet inlet : inlets) { for (int index = 0; index < inlets.size(); index++) {
int box = 4 * index;
if (x < inletBoxes[box] || z < inletBoxes[box + 1] || x > inletBoxes[box + 2] || z > inletBoxes[box + 3]) {
continue;
}
SeaInlet inlet = inlets.get(index);
if (SeaInletRasterizer.column(inlet, x, z, column) && column[1] == SeaInlet.OPEN) { if (SeaInletRasterizer.column(inlet, x, z, column) && column[1] == SeaInlet.OPEN) {
return inlet; return inlet;
} }
@@ -179,6 +191,8 @@ final class ChunkFeatures {
for (Reach reach : reaches) { for (Reach reach : reaches) {
segments.addNear(reach, minX - 1f, minZ - 1f, maxX + 1f, maxZ + 1f, outward); segments.addNear(reach, minX - 1f, minZ - 1f, maxX + 1f, maxZ + 1f, outward);
} }
int riffle = RiffleProfile.RADIUS + 1;
segments.coverChannels(minX - riffle, minZ - riffle, maxX + riffle, maxZ + riffle);
HydrologySettings.PondShore shore = settings.ponds().shore(); HydrologySettings.PondShore shore = settings.ponds().shore();
int reach = Math.max(shore.taper(), ChunkWaterRaster.MAX_SHORE_DISTANCE) + 1; int reach = Math.max(shore.taper(), ChunkWaterRaster.MAX_SHORE_DISTANCE) + 1;
int shelf = shore.shelf() + 1; int shelf = shore.shelf() + 1;
@@ -192,6 +206,39 @@ final class ChunkFeatures {
lakeReach.collect(mask, minX - shoreReach, minZ - shoreReach, maxX + shoreReach, maxZ + shoreReach); lakeReach.collect(mask, minX - shoreReach, minZ - shoreReach, maxX + shoreReach, maxZ + shoreReach);
System.arraycopy(grownBox(mask), 0, lakeBoxes, 4 * index, 4); System.arraycopy(grownBox(mask), 0, lakeBoxes, 4 * index, 4);
} }
inert = segments.isEmpty() && !inletsTouch(minX - 1, minZ - 1, maxX + 1, maxZ + 1)
&& !pondsTouch(minX - 1 - reach, minZ - 1 - reach, maxX + 1 + reach, maxZ + 1 + reach)
&& !lakesTouch(minX - 2 - shoreReach, minZ - 2 - shoreReach, maxX + 2 + shoreReach, maxZ + 2 + shoreReach);
}
private boolean inletsTouch(int minX, int minZ, int maxX, int maxZ) {
for (int box = 0; box < inletBoxes.length; box += 4) {
if (inletBoxes[box] <= maxX && inletBoxes[box + 2] >= minX && inletBoxes[box + 1] <= maxZ && inletBoxes[box + 3] >= minZ) {
return true;
}
}
return false;
}
private boolean pondsTouch(int minX, int minZ, int maxX, int maxZ) {
for (Pond pond : ponds) {
PondMask mask = pond.mask();
int fromX = mask.originX() - (mask.step() >> 1);
int fromZ = mask.originZ() - (mask.step() >> 1);
if (fromX <= maxX && fromX + mask.width() * mask.step() > minX && fromZ <= maxZ && fromZ + mask.depth() * mask.step() > minZ) {
return true;
}
}
return false;
}
private boolean lakesTouch(int minX, int minZ, int maxX, int maxZ) {
for (int box = 0; box < lakeBoxes.length; box += 4) {
if (lakeBoxes[box] <= maxX && lakeBoxes[box + 2] > minX && lakeBoxes[box + 1] <= maxZ && lakeBoxes[box + 3] > minZ) {
return true;
}
}
return false;
} }
private static Map<WindowKey, IndexedWindow> windowsByKey(int chunkX, int chunkZ, List<IndexedWindow> windows) { private static Map<WindowKey, IndexedWindow> windowsByKey(int chunkX, int chunkZ, List<IndexedWindow> windows) {
@@ -8,17 +8,18 @@ import art.arcane.iris.generation.hydrology.water.HydrologyTier;
import java.util.List; import java.util.List;
import java.util.Objects; import java.util.Objects;
import java.util.function.IntBinaryOperator;
public final class ChunkRasterizer { public final class ChunkRasterizer {
private static final int CHUNK = 16; private static final int CHUNK = 16;
private static final int CANDIDATE_HALO = 48; private static final int CANDIDATE_HALO = 48;
private final HydrologySettings settings; private final HydrologySettings settings;
private final ChannelFluids channelFluids;
private final int halo; private final int halo;
public ChunkRasterizer(HydrologySettings settings) { public ChunkRasterizer(HydrologySettings settings, ChannelFluids channelFluids) {
this.settings = Objects.requireNonNull(settings, "settings"); this.settings = Objects.requireNonNull(settings, "settings");
this.channelFluids = Objects.requireNonNull(channelFluids, "channelFluids");
float widest = 0f; float widest = 0f;
for (HydrologyTier tier : settings.tiers()) { for (HydrologyTier tier : settings.tiers()) {
widest = Math.max(widest, tier.maxHalfWidth()); widest = Math.max(widest, tier.maxHalfWidth());
@@ -33,13 +34,13 @@ public final class ChunkRasterizer {
chunkX * CHUNK + CHUNK - 1 + halo, chunkZ * CHUNK + CHUNK - 1 + halo); chunkX * CHUNK + CHUNK - 1 + halo, chunkZ * CHUNK + CHUNK - 1 + halo);
} }
public ChunkWaterRaster rasterize(int chunkX, int chunkZ, IntBinaryOperator naturalHeight, List<IndexedWindow> windows, public ChunkWaterRaster rasterize(int chunkX, int chunkZ, NaturalHeights naturalHeight, List<IndexedWindow> windows,
List<RegionPlan> lakeRegionPlans) { List<RegionPlan> lakeRegionPlans) {
Objects.requireNonNull(naturalHeight, "naturalHeight"); Objects.requireNonNull(naturalHeight, "naturalHeight");
ChunkFeatures features = ChunkFeatures.gather(settings, chunkX, chunkZ, halo, windows, lakeRegionPlans, lakeRegions(chunkX, chunkZ)); ChunkFeatures features = ChunkFeatures.gather(settings, chunkX, chunkZ, halo, windows, lakeRegionPlans, lakeRegions(chunkX, chunkZ));
if (features.empty()) { if (features.inert()) {
return ChunkWaterRaster.EMPTY; return ChunkWaterRaster.EMPTY;
} }
return new RasterComposer(settings, features, chunkX, chunkZ).compose(naturalHeight); return new RasterComposer(settings, features, channelFluids, chunkX, chunkZ).compose(naturalHeight);
} }
} }
@@ -2,6 +2,8 @@ package art.arcane.iris.generation.hydrology.raster;
import art.arcane.iris.generation.hydrology.water.HydrologyTier; import art.arcane.iris.generation.hydrology.water.HydrologyTier;
import java.util.Arrays;
public final class ChunkWaterRaster { public final class ChunkWaterRaster {
public static final int MAX_SHORE_DISTANCE = 32; public static final int MAX_SHORE_DISTANCE = 32;
public static final ChunkWaterRaster PROVISIONAL = new ChunkWaterRaster(true); public static final ChunkWaterRaster PROVISIONAL = new ChunkWaterRaster(true);
@@ -13,10 +15,13 @@ public final class ChunkWaterRaster {
private final int[] terrain; private final int[] terrain;
private final int[] head; private final int[] head;
private final byte[] role; private final byte[] role;
private final HydrologyTier[] tier; private final byte[] tier;
private final long[] featureId; private final HydrologyTier[] tiers;
private final String[] fluid; private final byte[] feature;
private final int[] caveCeiling; private final long[] features;
private final byte[] fluid;
private final String[] fluids;
private final boolean caveMask;
private final byte[] shoreDistance; private final byte[] shoreDistance;
private ChunkWaterRaster(boolean provisional) { private ChunkWaterRaster(boolean provisional) {
@@ -25,26 +30,32 @@ public final class ChunkWaterRaster {
this.head = null; this.head = null;
this.role = null; this.role = null;
this.tier = null; this.tier = null;
this.featureId = null; this.tiers = null;
this.feature = null;
this.features = null;
this.fluid = null; this.fluid = null;
this.caveCeiling = null; this.fluids = null;
this.caveMask = false;
this.shoreDistance = null; this.shoreDistance = null;
} }
ChunkWaterRaster(int[] terrain, int[] head, byte[] role, HydrologyTier[] tier, long[] featureId, String[] fluid, int[] caveCeiling, ChunkWaterRaster(int[] terrain, int[] head, byte[] role, Palette<HydrologyTier> tiers, LongPalette features, Palette<String> fluids,
byte[] shoreDistance) { boolean caveMask, byte[] shoreDistance) {
if (terrain.length != COLUMNS || head.length != COLUMNS || role.length != COLUMNS || tier.length != COLUMNS if (terrain.length != COLUMNS || head.length != COLUMNS || role.length != COLUMNS || tiers.indices.length != COLUMNS
|| featureId.length != COLUMNS || fluid.length != COLUMNS || caveCeiling.length != COLUMNS || shoreDistance.length != COLUMNS) { || features.indices.length != COLUMNS || fluids.indices.length != COLUMNS || shoreDistance.length != COLUMNS) {
throw new IllegalArgumentException("A chunk water raster needs " + COLUMNS + " entries per array"); throw new IllegalArgumentException("A chunk water raster needs " + COLUMNS + " entries per array");
} }
this.provisional = false; this.provisional = false;
this.terrain = terrain; this.terrain = terrain;
this.head = head; this.head = head;
this.role = role; this.role = role;
this.tier = tier; this.tier = tiers.indices;
this.featureId = featureId; this.tiers = tiers.values();
this.fluid = fluid; this.feature = features.indices;
this.caveCeiling = caveCeiling; this.features = features.values();
this.fluid = fluids.indices;
this.fluids = fluids.values();
this.caveMask = caveMask;
this.shoreDistance = shoreDistance; this.shoreDistance = shoreDistance;
} }
@@ -68,22 +79,22 @@ public final class ChunkWaterRaster {
public HydrologyTier tier(int lx, int lz) { public HydrologyTier tier(int lx, int lz) {
int index = index(lx, lz); int index = index(lx, lz);
return tier == null ? null : tier[index]; return tier == null ? null : tiers[tier[index] & 0xFF];
} }
public long featureId(int lx, int lz) { public long featureId(int lx, int lz) {
int index = index(lx, lz); int index = index(lx, lz);
return featureId == null ? 0L : featureId[index]; return feature == null ? 0L : features[feature[index] & 0xFF];
} }
public String fluid(int lx, int lz) { public String fluid(int lx, int lz) {
int index = index(lx, lz); int index = index(lx, lz);
return fluid == null ? null : fluid[index]; return fluid == null ? null : fluids[fluid[index] & 0xFF];
} }
public int caveCeiling(int lx, int lz) { public int caveCeiling(int lx, int lz) {
int index = index(lx, lz); int index = index(lx, lz);
return caveCeiling == null ? Integer.MAX_VALUE : caveCeiling[index]; return caveMask && head[index] != Integer.MIN_VALUE ? terrain[index] - 1 : Integer.MAX_VALUE;
} }
public int shoreDistance(int lx, int lz) { public int shoreDistance(int lx, int lz) {
@@ -105,4 +116,53 @@ public final class ChunkWaterRaster {
} }
return lz << 4 | lx; return lz << 4 | lx;
} }
static final class Palette<T> {
private final byte[] indices = new byte[COLUMNS];
private final T[] seen;
private int size;
Palette(T[] seen) {
if (seen.length != COLUMNS) {
throw new IllegalArgumentException("A raster palette needs room for " + COLUMNS + " values");
}
this.seen = seen;
}
void set(int column, T value) {
int found = 0;
while (found < size && seen[found] != value) {
found++;
}
if (found == size) {
seen[size++] = value;
}
indices[column] = (byte) found;
}
private T[] values() {
return Arrays.copyOf(seen, size);
}
}
static final class LongPalette {
private final byte[] indices = new byte[COLUMNS];
private final long[] seen = new long[COLUMNS];
private int size;
void set(int column, long value) {
int found = 0;
while (found < size && seen[found] != value) {
found++;
}
if (found == size) {
seen[size++] = value;
}
indices[column] = (byte) found;
}
private long[] values() {
return Arrays.copyOf(seen, size);
}
}
} }
@@ -0,0 +1,6 @@
package art.arcane.iris.generation.hydrology.raster;
@FunctionalInterface
public interface NaturalHeights {
double height(int x, int z);
}
@@ -2,7 +2,6 @@ package art.arcane.iris.generation.hydrology.raster;
import art.arcane.iris.generation.hydrology.config.IrisBankStyle; import art.arcane.iris.generation.hydrology.config.IrisBankStyle;
import art.arcane.iris.generation.hydrology.cave.HydrologyCaveCell; import art.arcane.iris.generation.hydrology.cave.HydrologyCaveCell;
import art.arcane.iris.generation.hydrology.grid.CellHash;
import art.arcane.iris.generation.hydrology.plan.HydrologySettings; import art.arcane.iris.generation.hydrology.plan.HydrologySettings;
import art.arcane.iris.generation.hydrology.underground.SeaInlet; import art.arcane.iris.generation.hydrology.underground.SeaInlet;
import art.arcane.iris.generation.hydrology.water.HydrologyTier; import art.arcane.iris.generation.hydrology.water.HydrologyTier;
@@ -11,7 +10,6 @@ import art.arcane.iris.generation.hydrology.water.Reach;
import art.arcane.iris.generation.hydrology.water.ReachStyle; import art.arcane.iris.generation.hydrology.water.ReachStyle;
import java.util.Arrays; import java.util.Arrays;
import java.util.function.IntBinaryOperator;
final class RasterComposer { final class RasterComposer {
private static final int CHUNK = 16; private static final int CHUNK = 16;
@@ -22,12 +20,15 @@ final class RasterComposer {
private static final double MAX_CONE_SLOPE = 2D; private static final double MAX_CONE_SLOPE = 2D;
private static final double CONE_EXPONENT = 0.8D; private static final double CONE_EXPONENT = 0.8D;
private static final double ROUNDING = 1e-6D; private static final double ROUNDING = 1e-6D;
private static final long SALT_BANK = 0x42414e4b4e5aL; private static final int SLOPE_REACH = 2;
private static final int RAY_STEP = 4;
private final HydrologySettings settings; private final HydrologySettings settings;
private final ChunkFeatures features; private final ChunkFeatures features;
private final ChannelFluids channelFluids;
private final int originX; private final int originX;
private final int originZ; private final int originZ;
private final double[] exact = new double[CELLS];
private final int[] natural = new int[CELLS]; private final int[] natural = new int[CELLS];
private final int[] terrain = new int[CELLS]; private final int[] terrain = new int[CELLS];
private final int[] head = new int[CELLS]; private final int[] head = new int[CELLS];
@@ -41,11 +42,12 @@ final class RasterComposer {
private final HydrologyTier[] nearTier = new HydrologyTier[CELLS]; private final HydrologyTier[] nearTier = new HydrologyTier[CELLS];
private final ReachCursor cursor = new ReachCursor(); private final ReachCursor cursor = new ReachCursor();
private final int[] inletColumn = new int[2]; private final int[] inletColumn = new int[2];
private IntBinaryOperator naturalHeight; private NaturalHeights naturalHeight;
RasterComposer(HydrologySettings settings, ChunkFeatures features, int chunkX, int chunkZ) { RasterComposer(HydrologySettings settings, ChunkFeatures features, ChannelFluids channelFluids, int chunkX, int chunkZ) {
this.settings = settings; this.settings = settings;
this.features = features; this.features = features;
this.channelFluids = channelFluids;
this.originX = chunkX * CHUNK - 1; this.originX = chunkX * CHUNK - 1;
this.originZ = chunkZ * CHUNK - 1; this.originZ = chunkZ * CHUNK - 1;
Arrays.fill(head, Integer.MIN_VALUE); Arrays.fill(head, Integer.MIN_VALUE);
@@ -54,11 +56,12 @@ final class RasterComposer {
Arrays.fill(distance, Double.POSITIVE_INFINITY); Arrays.fill(distance, Double.POSITIVE_INFINITY);
} }
ChunkWaterRaster compose(IntBinaryOperator naturalHeight) { ChunkWaterRaster compose(NaturalHeights naturalHeight) {
this.naturalHeight = naturalHeight; this.naturalHeight = naturalHeight;
for (int cell = 0; cell < CELLS; cell++) { for (int cell = 0; cell < CELLS; cell++) {
if (!corner(cell)) { if (!corner(cell)) {
natural[cell] = naturalHeight.applyAsInt(originX + cell % GRID, originZ + cell / GRID); exact[cell] = naturalHeight.height(originX + cell % GRID, originZ + cell / GRID);
natural[cell] = (int) Math.round(exact[cell]);
terrain[cell] = natural[cell]; terrain[cell] = natural[cell];
} }
} }
@@ -107,7 +110,7 @@ final class RasterComposer {
return; return;
} }
if (cursor.channel.reach != null) { if (cursor.channel.reach != null) {
channel(cell, ground, variation, BedProfile.detail(settings.seed(), x, z)); channel(cell, x, z, ground, variation);
} else if (interior(cell)) { } else if (interior(cell)) {
dry(cell, x, z, ground, variation); dry(cell, x, z, ground, variation);
} }
@@ -129,16 +132,17 @@ final class RasterComposer {
return Math.min(floor, bed(hit, Math.min(level, level(hit.reach.grade()[hit.point()])), ground, variation, detail)); return Math.min(floor, bed(hit, Math.min(level, level(hit.reach.grade()[hit.point()])), ground, variation, detail));
} }
private void channel(int cell, int ground, double variation, double detail) { private void channel(int cell, int x, int z, int ground, double variation) {
ReachCursor.Choice hit = cursor.channel; ReachCursor.Choice hit = cursor.channel;
Reach reach = hit.reach; Reach reach = hit.reach;
int point = hit.point(); int point = hit.point();
String area = channelFluids.channelFluid(x, z);
head[cell] = level(reach.grade()[point]); head[cell] = level(reach.grade()[point]);
terrain[cell] = bed(hit, head[cell], ground, variation, detail); terrain[cell] = bed(hit, head[cell], ground, variation, BedProfile.detail(settings.seed(), x, z));
role[cell] = role(reach.style()[point]); role[cell] = role(reach.style()[point]);
feature[cell] = reach.id(); feature[cell] = reach.id();
tier[cell] = reach.tier(); tier[cell] = reach.tier();
fluid[cell] = reach.tier().fluid(); fluid[cell] = area != null ? area : reach.tier().fluid();
} }
private static int bed(ReachCursor.Choice hit, int surface, int ground, double variation, double detail) { private static int bed(ReachCursor.Choice hit, int surface, int ground, double variation, double detail) {
@@ -162,7 +166,8 @@ final class RasterComposer {
u = BedProfile.position(u, -variation); u = BedProfile.position(u, -variation);
} }
if (ground >= surface) { if (ground >= surface) {
cut = BankProfile.cut(bankTier.bankStyle(), surface, ground, u, coneSlope(cell) * u * bankTier.bankBand(), noise(x, z)); double cone = coneSlope(steepestToWater(x, z, bank, bankEdge)) * u * bankTier.bankBand();
cut = BankProfile.cut(bankTier.bankStyle(), surface, ground, u, cone, noise(x, z));
} else { } else {
raise = BankProfile.levee(bankTier.bankStyle(), surface, ground, u, noise(x, z), settings.deformation().maxFill()); raise = BankProfile.levee(bankTier.bankStyle(), surface, ground, u, noise(x, z), settings.deformation().maxFill());
} }
@@ -205,15 +210,47 @@ final class RasterComposer {
distance[cell] = best; distance[cell] = best;
} }
private double coneSlope(int cell) { private double coneSlope(double slope) {
double slope = Math.max(Math.abs(natural[cell + 1] - natural[cell - 1]), Math.abs(natural[cell + GRID] - natural[cell - GRID])) / 2D;
HydrologySettings.Deformation deformation = settings.deformation(); HydrologySettings.Deformation deformation = settings.deformation();
return Math.min(MAX_CONE_SLOPE, deformation.bankSlope() + deformation.bankSteepFactor() * Math.pow(slope, CONE_EXPONENT)); return Math.min(MAX_CONE_SLOPE, deformation.bankSlope() + deformation.bankSteepFactor() * Math.pow(slope, CONE_EXPONENT));
} }
private double steepestToWater(int x, int z, ReachCursor.Choice bank, double bankEdge) {
Reach reach = bank.reach;
int next = Math.min(bank.segment + 1, reach.xs().length - 1);
double centreX = reach.xs()[bank.segment] + (reach.xs()[next] - reach.xs()[bank.segment]) * bank.t;
double centreZ = reach.zs()[bank.segment] + (reach.zs()[next] - reach.zs()[bank.segment]) * bank.t;
double steepest = slopeAt(x, z);
double length = Math.hypot(centreX - x, centreZ - z);
if (length <= 0D) {
return steepest;
}
double towardX = (centreX - x) / length;
double towardZ = (centreZ - z) / length;
for (double edge = 0D; edge < bankEdge; edge += RAY_STEP) {
double back = bankEdge - edge;
steepest = Math.max(steepest, slopeAt((int) Math.floor(x + towardX * back + 0.5D), (int) Math.floor(z + towardZ * back + 0.5D)));
}
return steepest;
}
private double slopeAt(int x, int z) {
return Math.max(Math.abs(exactAt(x + SLOPE_REACH, z) - exactAt(x - SLOPE_REACH, z)),
Math.abs(exactAt(x, z + SLOPE_REACH) - exactAt(x, z - SLOPE_REACH))) / (2D * SLOPE_REACH);
}
private double exactAt(int x, int z) {
int gx = x - originX;
int gz = z - originZ;
if (gx >= 0 && gz >= 0 && gx < GRID && gz < GRID && !corner(gz * GRID + gx)) {
return exact[gz * GRID + gx];
}
return naturalHeight.height(x, z);
}
private double noise(int x, int z) { private double noise(int x, int z) {
float strength = settings.deformation().bankNoise(); float strength = settings.deformation().bankNoise();
return strength == 0f ? 0D : (CellHash.unit(CellHash.mix(settings.seed(), SALT_BANK, x, z)) - 0.5D) * strength; return strength == 0f ? 0D : BedProfile.roughness(settings.seed(), x, z) * strength;
} }
private static int taperCut(int level, int ground, double u) { private static int taperCut(int level, int ground, double u) {
@@ -349,7 +386,7 @@ final class RasterComposer {
if (gx >= 0 && gz >= 0 && gx < GRID && gz < GRID && !corner(gz * GRID + gx)) { if (gx >= 0 && gz >= 0 && gx < GRID && gz < GRID && !corner(gz * GRID + gx)) {
return natural[gz * GRID + gx]; return natural[gz * GRID + gx];
} }
return naturalHeight.applyAsInt(x, z); return (int) Math.round(naturalHeight.height(x, z));
} }
private void nameShores() { private void nameShores() {
@@ -385,34 +422,37 @@ final class RasterComposer {
private ChunkWaterRaster pack() { private ChunkWaterRaster pack() {
int columns = ChunkWaterRaster.COLUMNS; int columns = ChunkWaterRaster.COLUMNS;
boolean affected = false;
for (int lz = 0; lz < CHUNK && !affected; lz++) {
for (int lx = 0; lx < CHUNK && !affected; lx++) {
int cell = (lz + 1) * GRID + lx + 1;
affected = role[cell] != WaterRole.NONE || terrain[cell] != natural[cell] || shore(cell) < ChunkWaterRaster.MAX_SHORE_DISTANCE;
}
}
if (!affected) {
return ChunkWaterRaster.EMPTY;
}
int[] terrainOut = new int[columns]; int[] terrainOut = new int[columns];
int[] headOut = new int[columns]; int[] headOut = new int[columns];
byte[] roleOut = new byte[columns]; byte[] roleOut = new byte[columns];
HydrologyTier[] tierOut = new HydrologyTier[columns]; ChunkWaterRaster.Palette<HydrologyTier> tierOut = new ChunkWaterRaster.Palette<>(new HydrologyTier[columns]);
long[] featureOut = new long[columns]; ChunkWaterRaster.LongPalette featureOut = new ChunkWaterRaster.LongPalette();
String[] fluidOut = new String[columns]; ChunkWaterRaster.Palette<String> fluidOut = new ChunkWaterRaster.Palette<>(new String[columns]);
int[] caveOut = new int[columns];
byte[] shoreOut = new byte[columns]; byte[] shoreOut = new byte[columns];
boolean caveMask = settings.deformation().caveMask();
boolean affected = false;
for (int lz = 0; lz < CHUNK; lz++) { for (int lz = 0; lz < CHUNK; lz++) {
for (int lx = 0; lx < CHUNK; lx++) { for (int lx = 0; lx < CHUNK; lx++) {
int cell = (lz + 1) * GRID + lx + 1; int cell = (lz + 1) * GRID + lx + 1;
int index = lz << 4 | lx; int index = lz << 4 | lx;
int shore = shore(cell);
terrainOut[index] = terrain[cell]; terrainOut[index] = terrain[cell];
headOut[index] = head[cell]; headOut[index] = head[cell];
roleOut[index] = (byte) role[cell].ordinal(); roleOut[index] = (byte) role[cell].ordinal();
tierOut[index] = tier[cell]; tierOut.set(index, tier[cell]);
featureOut[index] = feature[cell]; featureOut.set(index, feature[cell]);
fluidOut[index] = fluid[cell]; fluidOut.set(index, fluid[cell]);
caveOut[index] = caveMask && head[cell] != Integer.MIN_VALUE ? terrain[cell] - 1 : Integer.MAX_VALUE; shoreOut[index] = (byte) shore(cell);
shoreOut[index] = (byte) shore;
affected |= role[cell] != WaterRole.NONE || terrain[cell] != natural[cell] || shore < ChunkWaterRaster.MAX_SHORE_DISTANCE;
} }
} }
return affected ? new ChunkWaterRaster(terrainOut, headOut, roleOut, tierOut, featureOut, fluidOut, caveOut, shoreOut) return new ChunkWaterRaster(terrainOut, headOut, roleOut, tierOut, featureOut, fluidOut, settings.deformation().caveMask(), shoreOut);
: ChunkWaterRaster.EMPTY;
} }
private int shore(int cell) { private int shore(int cell) {
@@ -6,6 +6,8 @@ import java.util.Arrays;
final class ReachSegments { final class ReachSegments {
private static final int RUN = 16; private static final int RUN = 16;
private static final int COVER_SHIFT = 2;
private static final float COVER_MARGIN = 1f;
private Reach[] reaches; private Reach[] reaches;
private int[] segments; private int[] segments;
@@ -24,6 +26,13 @@ final class ReachSegments {
private int[] runBands = new int[8]; private int[] runBands = new int[8];
private int runs; private int runs;
private boolean sealed; private boolean sealed;
private boolean covered;
private int coverX;
private int coverZ;
private int coverWidth;
private int coverDepth;
private int[] coverStarts;
private int[] coverEntries;
ReachSegments(int capacity) { ReachSegments(int capacity) {
int initial = Math.max(1, capacity); int initial = Math.max(1, capacity);
@@ -71,6 +80,19 @@ final class ReachSegments {
} }
} }
boolean isEmpty() {
return size == 0;
}
void coverChannels(int minX, int minZ, int maxX, int maxZ) {
covered = true;
coverX = minX;
coverZ = minZ;
coverWidth = ((maxX - minX) >> COVER_SHIFT) + 1;
coverDepth = ((maxZ - minZ) >> COVER_SHIFT) + 1;
sealed = false;
}
void nearest(double x, double z, Scope scope, ReachCursor cursor) { void nearest(double x, double z, Scope scope, ReachCursor cursor) {
if (!sealed) { if (!sealed) {
seal(); seal();
@@ -89,6 +111,16 @@ final class ReachSegments {
hint(channelHint, x, z, unbounded, cursor); hint(channelHint, x, z, unbounded, cursor);
hint(bankHint, x, z, unbounded, cursor); hint(bankHint, x, z, unbounded, cursor);
hint(edgeHint, x, z, unbounded, cursor); hint(edgeHint, x, z, unbounded, cursor);
int cell = scope == Scope.CHANNEL ? coveredCell(x, z) : -1;
if (cell >= 0) {
for (int entry = coverStarts[cell]; entry < coverStarts[cell + 1]; entry++) {
int index = coverEntries[entry];
if (index != channelHint && index != bankHint && index != edgeHint && !beyondSegment(index, x, z, scope, cursor)) {
evaluate(index, x, z, false, cursor);
}
}
return;
}
int first = 0; int first = 0;
for (int run = 0; run < runs; run++) { for (int run = 0; run < runs; run++) {
int end = runEnds[run]; int end = runEnds[run];
@@ -99,9 +131,7 @@ final class ReachSegments {
continue; continue;
} }
for (int index = first; index < end; index++) { for (int index = first; index < end; index++) {
if (!beyond(x, z, Math.min(startX[index], endX[index]), Math.min(startZ[index], endZ[index]), Math.max(startX[index], endX[index]), if (!beyondSegment(index, x, z, scope, cursor) && index != channelHint && index != bankHint && index != edgeHint) {
Math.max(startZ[index], endZ[index]), limit(radius[index], bands[index], scope, cursor))
&& index != channelHint && index != bankHint && index != edgeHint) {
evaluate(index, x, z, unbounded, cursor); evaluate(index, x, z, unbounded, cursor);
} }
} }
@@ -109,6 +139,23 @@ final class ReachSegments {
} }
} }
private int coveredCell(double x, double z) {
if (coverStarts == null) {
return -1;
}
int cellX = ((int) Math.floor(x) - coverX) >> COVER_SHIFT;
int cellZ = ((int) Math.floor(z) - coverZ) >> COVER_SHIFT;
if (cellX < 0 || cellZ < 0 || cellX >= coverWidth || cellZ >= coverDepth) {
return -1;
}
return cellZ * coverWidth + cellX;
}
private boolean beyondSegment(int index, double x, double z, Scope scope, ReachCursor cursor) {
return beyond(x, z, Math.min(startX[index], endX[index]), Math.min(startZ[index], endZ[index]), Math.max(startX[index], endX[index]),
Math.max(startZ[index], endZ[index]), limit(radius[index], bands[index], scope, cursor));
}
private void hint(int index, double x, double z, boolean unbounded, ReachCursor cursor) { private void hint(int index, double x, double z, boolean unbounded, ReachCursor cursor) {
if (index >= 0 && index < size) { if (index >= 0 && index < size) {
evaluate(index, x, z, unbounded, cursor); evaluate(index, x, z, unbounded, cursor);
@@ -171,9 +218,56 @@ final class ReachSegments {
addRun(start, end); addRun(start, end);
start = end; start = end;
} }
if (covered) {
cover();
}
sealed = true; sealed = true;
} }
private void cover() {
int cells = coverWidth * coverDepth;
int[] starts = new int[cells + 1];
int[] range = new int[4];
for (int index = 0; index < size; index++) {
if (coverRange(index, range)) {
for (int cellZ = range[1]; cellZ <= range[3]; cellZ++) {
for (int cellX = range[0]; cellX <= range[2]; cellX++) {
starts[cellZ * coverWidth + cellX + 1]++;
}
}
}
}
for (int cell = 0; cell < cells; cell++) {
starts[cell + 1] += starts[cell];
}
int[] entries = new int[starts[cells]];
int[] fill = Arrays.copyOf(starts, cells);
for (int index = 0; index < size; index++) {
if (coverRange(index, range)) {
for (int cellZ = range[1]; cellZ <= range[3]; cellZ++) {
for (int cellX = range[0]; cellX <= range[2]; cellX++) {
entries[fill[cellZ * coverWidth + cellX]++] = index;
}
}
}
}
coverStarts = starts;
coverEntries = entries;
}
private boolean coverRange(int index, int[] range) {
float reach = radius[index] + COVER_MARGIN;
range[0] = Math.max(0, coverCell(Math.min(startX[index], endX[index]) - reach, coverX));
range[1] = Math.max(0, coverCell(Math.min(startZ[index], endZ[index]) - reach, coverZ));
range[2] = Math.min(coverWidth - 1, coverCell(Math.max(startX[index], endX[index]) + reach, coverX));
range[3] = Math.min(coverDepth - 1, coverCell(Math.max(startZ[index], endZ[index]) + reach, coverZ));
return range[0] <= range[2] && range[1] <= range[3];
}
private static int coverCell(float block, int origin) {
return ((int) Math.floor(block) - origin) >> COVER_SHIFT;
}
private void addRun(int start, int end) { private void addRun(int start, int end) {
if (runs == runEnds.length) { if (runs == runEnds.length) {
runEnds = Arrays.copyOf(runEnds, 2 * runs); runEnds = Arrays.copyOf(runEnds, 2 * runs);
@@ -15,6 +15,7 @@ final class RiffleProfile {
private static final int LIMIT = 10 * RADIUS; private static final int LIMIT = 10 * RADIUS;
private static final int DRY = Integer.MIN_VALUE; private static final int DRY = Integer.MIN_VALUE;
private static final int[] SIDES = {1, -1, GRID, -GRID}; private static final int[] SIDES = {1, -1, GRID, -GRID};
private static final int[] REMAINING = remaining();
private final List<Field> fields = new ArrayList<>(); private final List<Field> fields = new ArrayList<>();
@@ -85,6 +86,20 @@ final class RiffleProfile {
return (int) Math.max(original, Math.min(head - 1L, Math.round(raised))); return (int) Math.max(original, Math.min(head - 1L, Math.round(raised)));
} }
private static int[] remaining() {
int[] remaining = new int[GRID * GRID];
int low = PADDING - 1;
int high = PADDING + CHUNK;
for (int z = 0; z < GRID; z++) {
for (int x = 0; x < GRID; x++) {
int outsideX = Math.max(0, Math.max(low - x, x - high));
int outsideZ = Math.max(0, Math.max(low - z, z - high));
remaining[z * GRID + x] = 10 * Math.max(outsideX, outsideZ);
}
}
return remaining;
}
private record Field(int crest, int[] distances) { private record Field(int crest, int[] distances) {
} }
@@ -107,6 +122,9 @@ final class RiffleProfile {
} }
private void offer(int cell, int distance) { private void offer(int cell, int distance) {
if (distance + REMAINING[cell] >= LIMIT) {
return;
}
if (distances[cell] < LIMIT) { if (distances[cell] < LIMIT) {
remove(cell); remove(cell);
} }
@@ -10,6 +10,7 @@ import it.unimi.dsi.fastutil.longs.LongOpenHashSet;
import java.util.Arrays; import java.util.Arrays;
import java.util.Set; import java.util.Set;
import java.util.function.Supplier;
public final class CavePoolFiller { public final class CavePoolFiller {
public static final int REACH = HydrologySettings.MAX_CAVE_POOL_RADIUS + 1; public static final int REACH = HydrologySettings.MAX_CAVE_POOL_RADIUS + 1;
@@ -40,15 +41,19 @@ public final class CavePoolFiller {
this.density = settings.density(); this.density = settings.density();
} }
public void fill(int chunkX, int chunkZ, CarvedTerrain carved, MantleWriter writer, long seed) { public void fill(int chunkX, int chunkZ, Supplier<? extends CarvedTerrain> terrain, MantleWriter writer, long seed) {
if (!pools.enabled() || density == 0f) { if (!pools.enabled() || density == 0f) {
return; return;
} }
int attempts = attempts(seed, chunkX, chunkZ); int attempts = attempts(seed, chunkX, chunkZ);
CarvedTerrain carved = null;
for (int attempt = 0; attempt < attempts; attempt++) { for (int attempt = 0; attempt < attempts; attempt++) {
if (density < 1f && CellHash.unit(CellHash.mix(seed, SALT_DENSITY + attempt, chunkX, chunkZ)) >= density) { if (density < 1f && CellHash.unit(CellHash.mix(seed, SALT_DENSITY + attempt, chunkX, chunkZ)) >= density) {
continue; continue;
} }
if (carved == null) {
carved = terrain.get();
}
long column = CellHash.mix(seed, SALT_COLUMN + attempt, chunkX, chunkZ); long column = CellHash.mix(seed, SALT_COLUMN + attempt, chunkX, chunkZ);
int x = (chunkX << 4) + (int) (column & 15); int x = (chunkX << 4) + (int) (column & 15);
int z = (chunkZ << 4) + (int) ((column >>> 4) & 15); int z = (chunkZ << 4) + (int) ((column >>> 4) & 15);
@@ -33,6 +33,7 @@ public final class SeaInletPlanner {
private static final long SALT_HEADROOM = 0x53454148454144L; private static final long SALT_HEADROOM = 0x53454148454144L;
private static final long SALT_DEPTH = 0x53454144455054L; private static final long SALT_DEPTH = 0x53454144455054L;
private static final long SALT_DENSITY = 0x53454144454e53L; private static final long SALT_DENSITY = 0x53454144454e53L;
private static final double SQUARED_MARGIN = 1e-9D;
private final HydrologySettings settings; private final HydrologySettings settings;
@@ -403,22 +404,36 @@ public final class SeaInletPlanner {
double dx = bx - ax; double dx = bx - ax;
double dz = bz - az; double dz = bz - az;
double span = dx * dx + dz * dz; double span = dx * dx + dz * dz;
double outside = radius > 0f ? squaredAbove(radius) : Double.POSITIVE_INFINITY;
double inside = radius > 0f ? squaredBelow(radius) : Double.NEGATIVE_INFINITY;
double wetOutside = wetRadius > 0f ? squaredAbove(wetRadius) : Double.POSITIVE_INFINITY;
double wetInside = wetRadius > 0f ? squaredBelow(wetRadius) : Double.NEGATIVE_INFINITY;
for (int j = minJ; j <= maxJ; j++) { for (int j = minJ; j <= maxJ; j++) {
double pz = grid.originZ() + (double) j * cell; double pz = grid.originZ() + (double) j * cell;
for (int i = minI; i <= maxI; i++) { for (int i = minI; i <= maxI; i++) {
double px = grid.originX() + (double) i * cell; double px = grid.originX() + (double) i * cell;
double t = span <= 0D ? 0D : Math.clamp(((px - ax) * dx + (pz - az) * dz) / span, 0D, 1D); double t = span <= 0D ? 0D : Math.clamp(((px - ax) * dx + (pz - az) * dz) / span, 0D, 1D);
double distance = Math.hypot(px - (ax + t * dx), pz - (az + t * dz)); double ex = px - (ax + t * dx);
if (distance > radius) { double ez = pz - (az + t * dz);
double squared = ex * ex + ez * ez;
if (squared > outside || !(squared < inside) && Math.hypot(ex, ez) > radius) {
continue; continue;
} }
int index = j * grid.width() + i; int index = j * grid.width() + i;
cover[index] = Math.min(cover[index], bed); cover[index] = Math.min(cover[index], bed);
wet[index] |= distance <= wetRadius; wet[index] |= squared < wetInside || !(squared > wetOutside) && Math.hypot(ex, ez) <= wetRadius;
} }
} }
} }
private static double squaredAbove(float radius) {
return (double) radius * radius * (1D + SQUARED_MARGIN);
}
private static double squaredBelow(float radius) {
return (double) radius * radius * (1D - SQUARED_MARGIN);
}
private static float pathLength(DemGrid grid, List<Integer> path) { private static float pathLength(DemGrid grid, List<Integer> path) {
float length = 0f; float length = 0f;
for (int station = 1; station < path.size(); station++) { for (int station = 1; station < path.size(); station++) {
@@ -85,8 +85,9 @@ public final class MantleHydrologyComponent extends IrisMantleComponent {
generateFalls(writer, complex, runtime, own, chunkX, chunkZ); generateFalls(writer, complex, runtime, own, chunkX, chunkZ);
if (poolsActive()) { if (poolsActive()) {
HydrologySettings settings = runtime.settings(); HydrologySettings settings = runtime.settings();
new CavePoolFiller(settings).fill(chunkX, chunkZ, Engine engine = getEngineMantle().getEngine();
new MantleCarvedTerrain(writer, complex, getEngineMantle().getEngine(), chunkX, chunkZ), writer, settings.seed()); new CavePoolFiller(settings).fill(chunkX, chunkZ, () -> new MantleCarvedTerrain(writer, complex, engine, chunkX, chunkZ), writer,
settings.seed());
} }
} }
@@ -91,7 +91,6 @@ import art.arcane.volmlib.util.interpolation.NoiseBoundsProvider;
import art.arcane.volmlib.util.noise.CNG; import art.arcane.volmlib.util.noise.CNG;
import art.arcane.volmlib.util.stream.ProceduralStream; import art.arcane.volmlib.util.stream.ProceduralStream;
import art.arcane.volmlib.util.stream.interpolation.Interpolated; import art.arcane.volmlib.util.stream.interpolation.Interpolated;
import it.unimi.dsi.fastutil.longs.Long2DoubleOpenHashMap;
import lombok.AccessLevel; import lombok.AccessLevel;
import lombok.Data; import lombok.Data;
import lombok.EqualsAndHashCode; import lombok.EqualsAndHashCode;
@@ -120,10 +119,11 @@ import java.util.concurrent.atomic.AtomicLong;
import java.util.function.BiFunction; import java.util.function.BiFunction;
@Data @Data
@EqualsAndHashCode(exclude = {"data", "gridBoundsCache", "sharedCornerBounds", "sharedBoundsBiomes", "terrainProgram", "bakeCache", "regionSource", "baseBiomeSource", "caveSource", "rockSource", "fluidSource", "childStyleSeed", "inferredByType", "naturalHeightSource", "naturalTrueBiomeSource", "trueBiomeSource", "frozenInterpolators", "frozenGenerators", "inferredBiomeStreams", "imageMapRuntime", "terrainEngine", "terrain3D", "hydrologyRuntime", "shoreBiomes", "waterVegetation", "effectiveChunks"}) @EqualsAndHashCode(exclude = {"data", "gridBoundsCache", "sharedCornerBounds", "sharedBaseHeights", "sharedBoundsBiomes", "regionBounds", "contextRegionStream", "terrainProgram", "bakeCache", "regionSource", "baseBiomeSource", "caveSource", "rockSource", "fluidSource", "childStyleSeed", "inferredByType", "naturalHeightSource", "naturalTrueBiomeSource", "trueBiomeSource", "frozenInterpolators", "frozenGenerators", "inferredBiomeStreams", "imageMapRuntime", "terrainEngine", "terrain3D", "hydrologyRuntime", "shoreBiomes", "waterVegetation", "effectiveChunks"})
@ToString(exclude = {"data", "gridBoundsCache", "sharedCornerBounds", "sharedBoundsBiomes", "terrainProgram", "bakeCache", "regionSource", "baseBiomeSource", "caveSource", "rockSource", "fluidSource", "childStyleSeed", "inferredByType", "naturalHeightSource", "naturalTrueBiomeSource", "trueBiomeSource", "frozenInterpolators", "frozenGenerators", "inferredBiomeStreams", "imageMapRuntime", "terrainEngine", "terrain3D", "hydrologyRuntime", "shoreBiomes", "waterVegetation", "effectiveChunks"}) @ToString(exclude = {"data", "gridBoundsCache", "sharedCornerBounds", "sharedBaseHeights", "sharedBoundsBiomes", "regionBounds", "contextRegionStream", "terrainProgram", "bakeCache", "regionSource", "baseBiomeSource", "caveSource", "rockSource", "fluidSource", "childStyleSeed", "inferredByType", "naturalHeightSource", "naturalTrueBiomeSource", "trueBiomeSource", "frozenInterpolators", "frozenGenerators", "inferredBiomeStreams", "imageMapRuntime", "terrainEngine", "terrain3D", "hydrologyRuntime", "shoreBiomes", "waterVegetation", "effectiveChunks"})
public class IrisComplex implements DataProvider { public class IrisComplex implements DataProvider {
private static final NoiseBounds ZERO_NOISE_BOUNDS = new NoiseBounds(0D, 0D); private static final NoiseBounds ZERO_NOISE_BOUNDS = new NoiseBounds(0D, 0D);
private static final long NEGATIVE_ZERO_BITS = Double.doubleToRawLongBits(-0D);
private static final AtomicLong lastBoundsFailureLog = new AtomicLong(0L); private static final AtomicLong lastBoundsFailureLog = new AtomicLong(0L);
private static final int GRID_BOUNDS_CACHE_SIZE = 8192; private static final int GRID_BOUNDS_CACHE_SIZE = 8192;
private static final int STUDIO_NOISE_CACHE_SIZE = 32_768; private static final int STUDIO_NOISE_CACHE_SIZE = 32_768;
@@ -146,7 +146,6 @@ public class IrisComplex implements DataProvider {
Comparator.nullsFirst(Comparator.naturalOrder()) Comparator.nullsFirst(Comparator.naturalOrder())
); );
private static final ThreadLocal<TerrainBatchScratch> BATCH_SCRATCH = ThreadLocal.withInitial(TerrainBatchScratch::new); private static final ThreadLocal<TerrainBatchScratch> BATCH_SCRATCH = ThreadLocal.withInitial(TerrainBatchScratch::new);
private static final ThreadLocal<HydrologyBaseHeights> HYDROLOGY_BASE_HEIGHTS = new ThreadLocal<>();
private static final InferredType[] INFERRED_BIOME_PREPARATION_ORDER = { private static final InferredType[] INFERRED_BIOME_PREPARATION_ORDER = {
InferredType.LAND, InferredType.LAND,
InferredType.CAVE, InferredType.CAVE,
@@ -160,6 +159,7 @@ public class IrisComplex implements DataProvider {
@Getter(AccessLevel.NONE) @Getter(AccessLevel.NONE)
private final int biomeBoundsSamplingStep; private final int biomeBoundsSamplingStep;
private transient volatile SharedCornerBounds sharedCornerBounds; private transient volatile SharedCornerBounds sharedCornerBounds;
private transient volatile SharedBaseHeights sharedBaseHeights;
private transient volatile SharedBoundsBiomes sharedBoundsBiomes; private transient volatile SharedBoundsBiomes sharedBoundsBiomes;
@Getter(AccessLevel.NONE) @Getter(AccessLevel.NONE)
@Setter(AccessLevel.NONE) @Setter(AccessLevel.NONE)
@@ -259,6 +259,10 @@ public class IrisComplex implements DataProvider {
private IrisBiome focusBiome; private IrisBiome focusBiome;
private IrisRegion focusRegion; private IrisRegion focusRegion;
private Map<IrisInterpolator, IdentityHashMap<IrisBiome, GeneratorBounds>> generatorBounds; private Map<IrisInterpolator, IdentityHashMap<IrisBiome, GeneratorBounds>> generatorBounds;
@Getter(AccessLevel.NONE)
private final transient RegionBounds regionBounds;
@Getter(AccessLevel.NONE)
private final transient ProceduralStream<IrisRegion> contextRegionStream;
private Set<IrisBiome> generatorBiomes; private Set<IrisBiome> generatorBiomes;
@Setter(AccessLevel.NONE) @Setter(AccessLevel.NONE)
private final transient HydrologyRuntime hydrologyRuntime; private final transient HydrologyRuntime hydrologyRuntime;
@@ -386,6 +390,9 @@ public class IrisComplex implements DataProvider {
frozenGenerators[frozenIndex] = generatorGroups[frozenIndex].generators(); frozenGenerators[frozenIndex] = generatorGroups[frozenIndex].generators();
} }
generatorBounds = buildGeneratorBounds(engine); generatorBounds = buildGeneratorBounds(engine);
regionBounds = focusedBiomes == null && !imageMapRuntime.has(IrisImageMapApplication.BIOME)
? RegionBounds.compile(preparedRegions, frozenInterpolators.length, new BoundsReach(), this::groupBounds)
: null;
biomeBuffetGenerators = compileBuffetGeneratorGroups(); biomeBuffetGenerators = compileBuffetGeneratorGroups();
chunkHeightBatch = engine.getDimension().getTerrainSamplingStep() == 1 && biomeBuffetGenerators.isEmpty() chunkHeightBatch = engine.getDimension().getTerrainSamplingStep() == 1 && biomeBuffetGenerators.isEmpty()
&& (biomeBuffet == null || biomeBuffet.cells().isEmpty()); && (biomeBuffet == null || biomeBuffet.cells().isEmpty());
@@ -417,20 +424,21 @@ public class IrisComplex implements DataProvider {
: RegionSelection.select(new RegionSelection.Options( : RegionSelection.select(new RegionSelection.Options(
engine.getDimension(), data, regionStyleStream, imageMapRuntime, null)); engine.getDimension(), data, regionStyleStream, imageMapRuntime, null));
regionStream = focusedRegions != null ? focusedRegions : new BakedStream<>(bakeCache, BakePlane.REGION, regionSource); regionStream = focusedRegions != null ? focusedRegions : new BakedStream<>(bakeCache, BakePlane.REGION, regionSource);
caveSource = GenerationStreams.contextInjecting(regionStream, engine, (c, x, z) -> c.getRegion().get(x, z)) contextRegionStream = GenerationStreams.contextInjecting(regionStream, engine, (c, x, z) -> c.getRegion().get(x, z));
caveSource = contextRegionStream
.convert((r) -> createInferredBiomeStream(r, InferredType.CAVE)) .convert((r) -> createInferredBiomeStream(r, InferredType.CAVE))
.convertAware2D(ProceduralStream::get); .convertAware2D(ProceduralStream::get);
caveBiomeStream = new BakedStream<>(bakeCache, BakePlane.CAVE_BIOME, caveSource); caveBiomeStream = new BakedStream<>(bakeCache, BakePlane.CAVE_BIOME, caveSource);
inferredStreams.put(InferredType.CAVE, caveBiomeStream); inferredStreams.put(InferredType.CAVE, caveBiomeStream);
landBiomeStream = GenerationStreams.contextInjecting(regionStream, engine, (c, x, z) -> c.getRegion().get(x, z)) landBiomeStream = contextRegionStream
.convert((r) -> createInferredBiomeStream(r, InferredType.LAND)) .convert((r) -> createInferredBiomeStream(r, InferredType.LAND))
.convertAware2D(ProceduralStream::get); .convertAware2D(ProceduralStream::get);
inferredStreams.put(InferredType.LAND, landBiomeStream); inferredStreams.put(InferredType.LAND, landBiomeStream);
seaBiomeStream = GenerationStreams.contextInjecting(regionStream, engine, (c, x, z) -> c.getRegion().get(x, z)) seaBiomeStream = contextRegionStream
.convert((r) -> createInferredBiomeStream(r, InferredType.SEA)) .convert((r) -> createInferredBiomeStream(r, InferredType.SEA))
.convertAware2D(ProceduralStream::get); .convertAware2D(ProceduralStream::get);
inferredStreams.put(InferredType.SEA, seaBiomeStream); inferredStreams.put(InferredType.SEA, seaBiomeStream);
shoreBiomeStream = GenerationStreams.contextInjecting(regionStream, engine, (c, x, z) -> c.getRegion().get(x, z)) shoreBiomeStream = contextRegionStream
.convert((r) -> createInferredBiomeStream(r, InferredType.SHORE)) .convert((r) -> createInferredBiomeStream(r, InferredType.SHORE))
.convertAware2D(ProceduralStream::get); .convertAware2D(ProceduralStream::get);
inferredStreams.put(InferredType.SHORE, shoreBiomeStream); inferredStreams.put(InferredType.SHORE, shoreBiomeStream);
@@ -812,17 +820,35 @@ public class IrisComplex implements DataProvider {
} }
return; return;
} }
SharedBaseHeights memo = sharedBaseHeights();
TerrainBatchScratch scratch = BATCH_SCRATCH.get(); TerrainBatchScratch scratch = BATCH_SCRATCH.get();
int mark = scratch.mark(); int mark = scratch.mark();
int indexMark = scratch.indexMark(); int indexMark = scratch.indexMark();
try { try {
double[] batchXs = scratch.buffer(count); double[] batchXs = scratch.buffer(count);
double[] batchZs = scratch.buffer(count); double[] batchZs = scratch.buffer(count);
double[] computed = scratch.buffer(count);
int[] slots = scratch.indices(count);
int missing = 0;
for (int index = 0; index < count; index++) { for (int index = 0; index < count; index++) {
batchXs[index] = xs[from + index]; int x = xs[from + index];
batchZs[index] = zs[from + index]; int z = zs[from + index];
double known = memo.get(x, z);
if (Double.isNaN(known)) {
batchXs[missing] = x;
batchZs[missing] = z;
slots[missing++] = index;
} else {
heights[index] = known;
}
}
if (missing > 0) {
batchBaseHeights(scratch, batchXs, batchZs, missing, computed);
for (int index = 0; index < missing; index++) {
heights[slots[index]] = computed[index];
memo.put((int) batchXs[index], (int) batchZs[index], computed[index]);
}
} }
batchBaseHeights(scratch, batchXs, batchZs, count, heights);
} finally { } finally {
scratch.reset(mark); scratch.reset(mark);
scratch.resetIndices(indexMark); scratch.resetIndices(indexMark);
@@ -830,23 +856,9 @@ public class IrisComplex implements DataProvider {
if (baseOnly || terrain3D == null && transitionDisplacement == null) { if (baseOnly || terrain3D == null && transitionDisplacement == null) {
return; return;
} }
Long2DoubleOpenHashMap known = new Long2DoubleOpenHashMap(count);
known.defaultReturnValue(Double.NaN);
for (int index = 0; index < count; index++) { for (int index = 0; index < count; index++) {
known.put(hydrologyColumnKey(xs[from + index], zs[from + index]), heights[index]); heights[index] = naturalHeightStream.getDouble(xs[from + index], zs[from + index]);
} }
HYDROLOGY_BASE_HEIGHTS.set(new HydrologyBaseHeights(this, known));
try {
for (int index = 0; index < count; index++) {
heights[index] = naturalHeightStream.getDouble(xs[from + index], zs[from + index]);
}
} finally {
HYDROLOGY_BASE_HEIGHTS.remove();
}
}
private static long hydrologyColumnKey(int x, int z) {
return ((long) x << 32) | (z & 0xFFFFFFFFL);
} }
private void bakeBaseHeights(int chunkX, int chunkZ, long[] missing, double[] values) { private void bakeBaseHeights(int chunkX, int chunkZ, long[] missing, double[] values) {
@@ -1082,15 +1094,21 @@ public class IrisComplex implements DataProvider {
} }
private double sampleUnblendedNaturalTerrainHeight(double x, double z) { private double sampleUnblendedNaturalTerrainHeight(double x, double z) {
HydrologyBaseHeights batched = HYDROLOGY_BASE_HEIGHTS.get(); if (!memoColumn(x) || !memoColumn(z)) {
if (batched != null && batched.owner() == this && (int) x == x && (int) z == z) { return imageMapRuntime.sampleTerrainHeight(x, z, proceduralTerrainHeight.sample(x, z));
double known = batched.heights().get(hydrologyColumnKey((int) x, (int) z));
if (!Double.isNaN(known)) {
return known;
}
} }
double proceduralHeight = proceduralTerrainHeight.sample(x, z); SharedBaseHeights memo = sharedBaseHeights();
return imageMapRuntime.sampleTerrainHeight(x, z, proceduralHeight); double known = memo.get((int) x, (int) z);
if (!Double.isNaN(known)) {
return known;
}
double height = imageMapRuntime.sampleTerrainHeight(x, z, proceduralTerrainHeight.sample(x, z));
memo.put((int) x, (int) z, height);
return height;
}
private static boolean memoColumn(double coordinate) {
return (int) coordinate == coordinate && Double.doubleToRawLongBits(coordinate) != NEGATIVE_ZERO_BITS;
} }
static double calculateNaturalSlope(double naturalHeight, double easternHeight, double southernHeight) { static double calculateNaturalSlope(double naturalHeight, double easternHeight, double southernHeight) {
@@ -1662,7 +1680,11 @@ public class IrisComplex implements DataProvider {
/** The natural terrain height of a column, from the natural height stream only: never touches hydrology or its caches. */ /** The natural terrain height of a column, from the natural height stream only: never touches hydrology or its caches. */
public int naturalTrueHeight(int x, int z) { public int naturalTrueHeight(int x, int z) {
return (int) Math.round(naturalHeightStream.getDouble(x, z)); return (int) Math.round(naturalHeight(x, z));
}
public double naturalHeight(int x, int z) {
return naturalHeightStream.getDouble(x, z);
} }
/** The terrain slope of a column from the natural height stream only: never touches hydrology or its caches. */ /** The terrain slope of a column from the natural height stream only: never touches hydrology or its caches. */
@@ -1885,7 +1907,7 @@ public class IrisComplex implements DataProvider {
int grid = biomeBoundsSamplingStep; int grid = biomeBoundsSamplingStep;
GridBoundsCache cache = gridBoundsCache.get(); GridBoundsCache cache = gridBoundsCache.get();
if (grid <= 1) { if (grid <= 1) {
return sampleBoundsRaw(cache, engine, interpolator, generators, x, z); return sampleBoundsRaw(cache, engine, interpolator, interpolatorIndex, generators, x, z);
} }
int xi = (int) Math.floor(x); int xi = (int) Math.floor(x);
@@ -1941,7 +1963,7 @@ public class IrisComplex implements DataProvider {
if (GenerationCounter.ENABLED) { if (GenerationCounter.ENABLED) {
GenerationCounter.CORNER_STARCASTS.increment(); GenerationCounter.CORNER_STARCASTS.increment();
} }
packed = computePackedBounds(cache, engine, interpolator, generators, gx, gz); packed = computePackedBounds(cache, engine, interpolator, interpolatorIndex, generators, gx, gz);
if (!Double.isFinite(boundsLow(packed)) || !Double.isFinite(boundsHigh(packed))) { if (!Double.isFinite(boundsLow(packed)) || !Double.isFinite(boundsHigh(packed))) {
// A non-finite corner is a transient sampling fault, not a fact about the terrain: // A non-finite corner is a transient sampling fault, not a fact about the terrain:
// hand it back uncached so the next sample recomputes it instead of pinning it. // hand it back uncached so the next sample recomputes it instead of pinning it.
@@ -1972,6 +1994,20 @@ public class IrisComplex implements DataProvider {
return shared; return shared;
} }
private SharedBaseHeights sharedBaseHeights() {
SharedBaseHeights shared = sharedBaseHeights;
if (shared == null) {
synchronized (this) {
shared = sharedBaseHeights;
if (shared == null) {
shared = new SharedBaseHeights(SharedBaseHeights.capacityFor(Runtime.getRuntime().maxMemory()));
sharedBaseHeights = shared;
}
}
}
return shared;
}
private SharedBoundsBiomes sharedBoundsBiomes(ProceduralStream<IrisBiome> source) { private SharedBoundsBiomes sharedBoundsBiomes(ProceduralStream<IrisBiome> source) {
SharedBoundsBiomes shared = sharedBoundsBiomes; SharedBoundsBiomes shared = sharedBoundsBiomes;
if (shared != null && shared.source() == source) { if (shared != null && shared.source() == source) {
@@ -1987,8 +2023,9 @@ public class IrisComplex implements DataProvider {
} }
} }
private long computePackedBounds(GridBoundsCache cache, Engine engine, IrisInterpolator interpolator, IrisGenerator[] generators, int gx, int gz) { private long computePackedBounds(GridBoundsCache cache, Engine engine, IrisInterpolator interpolator, int interpolatorIndex,
NoiseBounds bounds = sampleBoundsRaw(cache, engine, interpolator, generators, gx, gz); IrisGenerator[] generators, int gx, int gz) {
NoiseBounds bounds = sampleBoundsRaw(cache, engine, interpolator, interpolatorIndex, generators, gx, gz);
return (((long) Float.floatToRawIntBits((float) bounds.min())) << 32) | (Float.floatToRawIntBits((float) bounds.max()) & 0xFFFFFFFFL); return (((long) Float.floatToRawIntBits((float) bounds.min())) << 32) | (Float.floatToRawIntBits((float) bounds.max()) & 0xFFFFFFFFL);
} }
@@ -2006,10 +2043,11 @@ public class IrisComplex implements DataProvider {
return a + ((b - a) * fz); return a + ((b - a) * fz);
} }
private NoiseBounds sampleBoundsRaw(GridBoundsCache cache, Engine engine, IrisInterpolator interpolator, IrisGenerator[] generators, double x, double z) { private NoiseBounds sampleBoundsRaw(GridBoundsCache cache, Engine engine, IrisInterpolator interpolator, int interpolatorIndex,
IrisGenerator[] generators, double x, double z) {
IdentityHashMap<IrisBiome, GeneratorBounds> cachedBounds = generatorBounds.get(interpolator); IdentityHashMap<IrisBiome, GeneratorBounds> cachedBounds = generatorBounds.get(interpolator);
BoundsSampler sampler = cache.sampler.isInUse() ? new BoundsSampler() : cache.sampler; BoundsSampler sampler = cache.sampler.isInUse() ? new BoundsSampler() : cache.sampler;
sampler.bind(this, engine, generators, cachedBounds); sampler.bind(this, engine, interpolatorIndex, generators, cachedBounds);
try { try {
return interpolator.interpolateBounds(x, z, sampler); return interpolator.interpolateBounds(x, z, sampler);
@@ -2297,6 +2335,11 @@ public class IrisComplex implements DataProvider {
return reached; return reached;
} }
private NoiseBounds groupBounds(int group, IrisBiome biome) {
GeneratorBounds cached = generatorBounds.get(frozenInterpolators[group]).get(biome);
return (cached == null ? computeGeneratorBounds(frozenGenerators[group], biome) : cached).noiseBounds;
}
static double averageGeneratorHeights( static double averageGeneratorHeights(
IrisGenerator[] generators, IrisGenerator[] generators,
double low, double low,
@@ -2604,6 +2647,32 @@ public class IrisComplex implements DataProvider {
} }
} }
private final class BoundsReach implements RegionBounds.Reach {
@Override
public List<IrisBiome> picks(IrisRegion region, InferredType type) {
return loadInferredBiomes(biomeKeys(region, type), type);
}
@Override
public Collection<IrisBiome> descendants(IrisBiome biome) {
Set<IrisBiome> reached = Collections.newSetFromMap(new IdentityHashMap<>());
ArrayDeque<IrisBiome> pending = new ArrayDeque<>();
pending.add(biome);
while (!pending.isEmpty()) {
IrisBiome next = pending.poll();
if (!reached.add(next)) {
continue;
}
for (IrisBiome child : next.getRealChildren(IrisComplex.this)) {
if (child != null) {
pending.add(child);
}
}
}
return reached;
}
}
/** /**
* Reusable per-thread bounds provider. Holds the biome memo and the lazily computed generator * Reusable per-thread bounds provider. Holds the biome memo and the lazily computed generator
* bounds for one sampleBoundsRaw pass so the pass allocates nothing; {@link #bind} * bounds for one sampleBoundsRaw pass so the pass allocates nothing; {@link #bind}
@@ -2619,9 +2688,12 @@ public class IrisComplex implements DataProvider {
private final IdentityHashMap<IrisBiome, GeneratorBounds> localBounds = new IdentityHashMap<>(8); private final IdentityHashMap<IrisBiome, GeneratorBounds> localBounds = new IdentityHashMap<>(8);
private IrisComplex complex; private IrisComplex complex;
private Engine engine; private Engine engine;
private int group;
private IrisGenerator[] generators; private IrisGenerator[] generators;
private IdentityHashMap<IrisBiome, GeneratorBounds> cachedBounds; private IdentityHashMap<IrisBiome, GeneratorBounds> cachedBounds;
private SharedBoundsBiomes sharedBiomes; private SharedBoundsBiomes sharedBiomes;
private BakedStream<IrisBiome> bakedBiomes;
private RegionBounds regional;
private boolean integerCoordinates; private boolean integerCoordinates;
private boolean inUse; private boolean inUse;
@@ -2629,14 +2701,19 @@ public class IrisComplex implements DataProvider {
return inUse; return inUse;
} }
private void bind(IrisComplex complex, Engine engine, IrisGenerator[] generators, IdentityHashMap<IrisBiome, GeneratorBounds> cachedBounds) { private void bind(IrisComplex complex, Engine engine, int group, IrisGenerator[] generators,
IdentityHashMap<IrisBiome, GeneratorBounds> cachedBounds) {
this.complex = complex; this.complex = complex;
this.engine = engine; this.engine = engine;
this.group = group;
this.generators = generators; this.generators = generators;
this.cachedBounds = cachedBounds; this.cachedBounds = cachedBounds;
this.integerCoordinates = complex.baseBiomeStream instanceof BakedStream<?> this.integerCoordinates = complex.baseBiomeStream instanceof BakedStream<?>
|| complex.baseBiomeStream instanceof CachedStream2D<?>; || complex.baseBiomeStream instanceof CachedStream2D<?>;
this.sharedBiomes = integerCoordinates ? complex.sharedBoundsBiomes(complex.baseBiomeStream) : null; this.sharedBiomes = integerCoordinates ? complex.sharedBoundsBiomes(complex.baseBiomeStream) : null;
this.bakedBiomes = complex.regionBounds != null && complex.baseBiomeStream instanceof BakedStream<IrisBiome> baked
? baked : null;
this.regional = bakedBiomes == null ? null : complex.regionBounds;
this.inUse = true; this.inUse = true;
if (sampleSource.get() != complex.baseBiomeStream) { if (sampleSource.get() != complex.baseBiomeStream) {
sampleCache.clear(); sampleCache.clear();
@@ -2654,6 +2731,8 @@ public class IrisComplex implements DataProvider {
generators = null; generators = null;
cachedBounds = null; cachedBounds = null;
sharedBiomes = null; sharedBiomes = null;
bakedBiomes = null;
regional = null;
inUse = false; inUse = false;
} }
@@ -2662,25 +2741,23 @@ public class IrisComplex implements DataProvider {
try { try {
double sampleX = integerCoordinates ? (int) xx : xx; double sampleX = integerCoordinates ? (int) xx : xx;
double sampleZ = integerCoordinates ? (int) zz : zz; double sampleZ = integerCoordinates ? (int) zz : zz;
IrisBiome bx = sampleCache.get(sampleX, sampleZ); Object known = sampleCache.get(sampleX, sampleZ);
if (bx == null) { if (known == null && sharedBiomes != null) {
bx = sharedBiomes == null ? null : sharedBiomes.get((int) sampleX, (int) sampleZ); known = sharedBiomes.get((int) sampleX, (int) sampleZ);
if (bx == null) { if (known != null) {
ChunkBakeCache.enterSparse(); sampleCache.put(sampleX, sampleZ, known);
try {
bx = complex.baseBiomeStream.get(xx, zz);
} finally {
ChunkBakeCache.exitSparse();
}
if (sharedBiomes != null) {
sharedBiomes.put((int) sampleX, (int) sampleZ, bx);
}
} }
sampleCache.put(sampleX, sampleZ, bx);
} }
if (known instanceof IrisBiome biome) {
GeneratorBounds bounds = complex.resolveGeneratorBounds(engine, generators, bx, cachedBounds, localBounds); return bounds(biome);
return bounds.noiseBounds; }
ChunkBakeCache.enterSparse();
try {
return regional == null ? bounds(remember(sampleX, sampleZ, complex.baseBiomeStream.get(xx, zz)))
: regionalBounds(known, sampleX, sampleZ);
} finally {
ChunkBakeCache.exitSparse();
}
} catch (Throwable e) { } catch (Throwable e) {
GenerationFailures.rethrowEngineFailure(e); GenerationFailures.rethrowEngineFailure(e);
long now = System.currentTimeMillis(); long now = System.currentTimeMillis();
@@ -2696,6 +2773,49 @@ public class IrisComplex implements DataProvider {
return ZERO_NOISE_BOUNDS; return ZERO_NOISE_BOUNDS;
} }
private NoiseBounds regionalBounds(Object known, double x, double z) {
RegionBounds.Pick pick = known instanceof RegionBounds.Pick picked ? picked : null;
IrisRegion region = pick != null ? pick.region() : known instanceof IrisRegion cached ? cached : null;
if (region == null) {
IrisBiome baked = bakedBiomes.baked((int) x, (int) z);
if (baked != null) {
return bounds(remember(x, z, baked));
}
region = remember(x, z, complex.contextRegionStream.get(x, z));
}
RegionBounds.Constants constants = regional.of(region);
NoiseBounds constant = constants == null ? null : constants.any(group);
if (constant != null) {
return constant;
}
InferredType type = pick != null ? pick.type() : complex.bridgeStream.get(x, z);
constant = constants == null ? null : constants.typed(type, group);
if (constant != null) {
return constant;
}
IrisBiome picked = pick != null ? pick.biome() : complex.createInferredBiomeStream(region, type).get(x, z);
constant = regional.pick(picked, group);
if (constant != null) {
if (pick == null) {
remember(x, z, new RegionBounds.Pick(region, type, picked));
}
return constant;
}
return bounds(remember(x, z, complex.implode(picked, x, z)));
}
private NoiseBounds bounds(IrisBiome biome) {
return complex.resolveGeneratorBounds(engine, generators, biome, cachedBounds, localBounds).noiseBounds;
}
private <T> T remember(double x, double z, T sample) {
if (sharedBiomes != null) {
sharedBiomes.put((int) x, (int) z, sample);
}
sampleCache.put(x, z, sample);
return sample;
}
} }
/** /**
@@ -2705,13 +2825,13 @@ public class IrisComplex implements DataProvider {
private static final class CoordinateBiomeCache { private static final class CoordinateBiomeCache {
private final long[] xBits; private final long[] xBits;
private final long[] zBits; private final long[] zBits;
private final IrisBiome[] values; private final Object[] values;
private final int mask; private final int mask;
private CoordinateBiomeCache(int capacity) { private CoordinateBiomeCache(int capacity) {
xBits = new long[capacity]; xBits = new long[capacity];
zBits = new long[capacity]; zBits = new long[capacity];
values = new IrisBiome[capacity]; values = new Object[capacity];
mask = capacity - 1; mask = capacity - 1;
} }
@@ -2719,20 +2839,20 @@ public class IrisComplex implements DataProvider {
Arrays.fill(values, null); Arrays.fill(values, null);
} }
private IrisBiome get(double x, double z) { private Object get(double x, double z) {
long xb = Double.doubleToLongBits(x); long xb = Double.doubleToLongBits(x);
long zb = Double.doubleToLongBits(z); long zb = Double.doubleToLongBits(z);
int slot = mix(xb, zb) & mask; int slot = mix(xb, zb) & mask;
return xBits[slot] == xb && zBits[slot] == zb ? values[slot] : null; return xBits[slot] == xb && zBits[slot] == zb ? values[slot] : null;
} }
private void put(double x, double z, IrisBiome biome) { private void put(double x, double z, Object sample) {
long xb = Double.doubleToLongBits(x); long xb = Double.doubleToLongBits(x);
long zb = Double.doubleToLongBits(z); long zb = Double.doubleToLongBits(z);
int slot = mix(xb, zb) & mask; int slot = mix(xb, zb) & mask;
xBits[slot] = xb; xBits[slot] = xb;
zBits[slot] = zb; zBits[slot] = zb;
values[slot] = biome; values[slot] = sample;
} }
private int mix(long xb, long zb) { private int mix(long xb, long zb) {
@@ -2784,7 +2904,4 @@ public class IrisComplex implements DataProvider {
transitionDisplacement.clear(); transitionDisplacement.clear();
} }
} }
private record HydrologyBaseHeights(IrisComplex owner, Long2DoubleOpenHashMap heights) {
}
} }
@@ -0,0 +1,120 @@
package art.arcane.iris.generation.runtime;
import art.arcane.iris.generation.biome.IrisBiome;
import art.arcane.iris.generation.terrain.InferredType;
import art.arcane.iris.generation.terrain.IrisRegion;
import art.arcane.volmlib.util.interpolation.NoiseBounds;
import java.util.Collection;
import java.util.Collections;
import java.util.IdentityHashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
final class RegionBounds {
private final Map<IrisRegion, Constants> regions;
private final Map<IrisBiome, NoiseBounds[]> picks;
private RegionBounds(Map<IrisRegion, Constants> regions, Map<IrisBiome, NoiseBounds[]> picks) {
this.regions = regions;
this.picks = picks;
}
static RegionBounds compile(Collection<IrisRegion> regions, int groups, Reach reach, GroupBounds bounds) {
IdentityHashMap<IrisRegion, Constants> compiled = new IdentityHashMap<>(Math.max(16, regions.size() * 2));
IdentityHashMap<IrisBiome, NoiseBounds[]> picks = new IdentityHashMap<>();
for (IrisRegion region : regions) {
if (compiled.containsKey(region)) {
continue;
}
NoiseBounds[] land = typeConstants(reach.picks(region, InferredType.LAND), groups, reach, bounds, picks);
NoiseBounds[] sea = typeConstants(reach.picks(region, InferredType.SEA), groups, reach, bounds, picks);
NoiseBounds[] any = new NoiseBounds[groups];
for (int group = 0; group < groups; group++) {
any[group] = land[group] != null && sea[group] != null && sameBits(land[group], sea[group]) ? land[group] : null;
}
compiled.put(region, new Constants(any, land, sea));
}
return new RegionBounds(compiled, picks);
}
Constants of(IrisRegion region) {
return regions.get(region);
}
NoiseBounds pick(IrisBiome picked, int group) {
NoiseBounds[] constants = picks.get(picked);
return constants == null ? null : constants[group];
}
private static NoiseBounds[] typeConstants(List<IrisBiome> picked, int groups, Reach reach, GroupBounds bounds,
Map<IrisBiome, NoiseBounds[]> picks) {
if (picked.isEmpty()) {
return new NoiseBounds[groups];
}
Set<IrisBiome> reached = Collections.newSetFromMap(new IdentityHashMap<>());
for (IrisBiome biome : picked) {
Collection<IrisBiome> descendants = reach.descendants(biome);
reached.addAll(descendants);
if (!picks.containsKey(biome)) {
picks.put(biome, constants(descendants, groups, bounds));
}
}
return constants(reached, groups, bounds);
}
private static NoiseBounds[] constants(Collection<IrisBiome> biomes, int groups, GroupBounds bounds) {
NoiseBounds[] constants = new NoiseBounds[groups];
for (int group = 0; group < groups; group++) {
constants[group] = constant(biomes, group, bounds);
}
return constants;
}
private static NoiseBounds constant(Collection<IrisBiome> biomes, int group, GroupBounds bounds) {
NoiseBounds first = null;
for (IrisBiome biome : biomes) {
NoiseBounds next = bounds.of(group, biome);
if (first == null) {
first = next;
} else if (!sameBits(first, next)) {
return null;
}
}
return first;
}
private static boolean sameBits(NoiseBounds first, NoiseBounds second) {
return Double.doubleToRawLongBits(first.min()) == Double.doubleToRawLongBits(second.min())
&& Double.doubleToRawLongBits(first.max()) == Double.doubleToRawLongBits(second.max());
}
record Constants(NoiseBounds[] any, NoiseBounds[] land, NoiseBounds[] sea) {
NoiseBounds any(int group) {
return any[group];
}
NoiseBounds typed(InferredType type, int group) {
return switch (type) {
case LAND -> land[group];
case SEA -> sea[group];
case SHORE, CAVE -> null;
};
}
}
record Pick(IrisRegion region, InferredType type, IrisBiome biome) {
}
interface Reach {
List<IrisBiome> picks(IrisRegion region, InferredType type);
Collection<IrisBiome> descendants(IrisBiome biome);
}
@FunctionalInterface
interface GroupBounds {
NoiseBounds of(int group, IrisBiome biome);
}
}
@@ -0,0 +1,76 @@
package art.arcane.iris.generation.runtime;
import java.lang.invoke.MethodHandles;
import java.lang.invoke.VarHandle;
final class SharedBaseHeights {
static final int MINIMUM_CAPACITY = 1 << 16;
static final int MAXIMUM_CAPACITY = 1 << 21;
static final long BYTES_PER_ENTRY = 20L;
static final long HEAP_SHARE = 256L;
private static final VarHandle INTS = MethodHandles.arrayElementVarHandle(int[].class);
private static final VarHandle LONGS = MethodHandles.arrayElementVarHandle(long[].class);
private final long[] keys;
private final long[] values;
private final int[] stamps;
private final int mask;
SharedBaseHeights(int capacity) {
if (capacity < 2 || Integer.bitCount(capacity) != 1) {
throw new IllegalArgumentException("capacity must be a power of two of at least 2");
}
this.keys = new long[capacity];
this.values = new long[capacity];
this.stamps = new int[capacity];
this.mask = capacity - 1;
}
static int capacityFor(long maxMemory) {
long entries = Math.max(0L, maxMemory) / HEAP_SHARE / BYTES_PER_ENTRY;
long bounded = Math.max(MINIMUM_CAPACITY, Math.min(MAXIMUM_CAPACITY, entries));
return Integer.highestOneBit((int) bounded);
}
double get(int x, int z) {
long key = key(x, z);
int slot = slot(key);
int before = (int) INTS.getAcquire(stamps, slot);
if (before == 0 || (before & 1) != 0) {
return Double.NaN;
}
long storedKey = (long) LONGS.getAcquire(keys, slot);
long value = (long) LONGS.getAcquire(values, slot);
VarHandle.loadLoadFence();
int after = (int) INTS.getAcquire(stamps, slot);
if (before != after || storedKey != key) {
return Double.NaN;
}
return Double.longBitsToDouble(value);
}
void put(int x, int z, double height) {
if (!Double.isFinite(height)) {
return;
}
long key = key(x, z);
int slot = slot(key);
int stamp = (int) INTS.getAcquire(stamps, slot);
if ((stamp & 1) != 0 || !INTS.compareAndSet(stamps, slot, stamp, stamp + 1)) {
return;
}
LONGS.setRelease(keys, slot, key);
LONGS.setRelease(values, slot, Double.doubleToRawLongBits(height));
INTS.setRelease(stamps, slot, stamp + 2);
}
private int slot(long key) {
long mixed = (key ^ (key >>> 30)) * 0xBF58476D1CE4E5B9L;
mixed = (mixed ^ (mixed >>> 27)) * 0x94D049BB133111EBL;
return (int) ((mixed ^ (mixed >>> 31)) & mask);
}
private static long key(int x, int z) {
return ((long) x << 32) | (z & 0xFFFFFFFFL);
}
}
@@ -1,7 +1,5 @@
package art.arcane.iris.generation.runtime; package art.arcane.iris.generation.runtime;
import art.arcane.iris.generation.biome.IrisBiome;
import java.lang.invoke.MethodHandles; import java.lang.invoke.MethodHandles;
import java.lang.invoke.VarHandle; import java.lang.invoke.VarHandle;
import java.util.Objects; import java.util.Objects;
@@ -49,7 +47,7 @@ final class SharedBoundsBiomes {
return (int) ((mixed ^ (mixed >>> 31)) & mask); return (int) ((mixed ^ (mixed >>> 31)) & mask);
} }
IrisBiome get(int x, int z) { Object get(int x, int z) {
long key = key(x, z); long key = key(x, z);
int slot = slot(x, z); int slot = slot(x, z);
int before = (int) INTS.getAcquire(stamps, slot); int before = (int) INTS.getAcquire(stamps, slot);
@@ -63,11 +61,11 @@ final class SharedBoundsBiomes {
if (before != after || storedKey != key) { if (before != after || storedKey != key) {
return null; return null;
} }
return (IrisBiome) value; return value;
} }
void put(int x, int z, IrisBiome biome) { void put(int x, int z, Object sample) {
if (biome == null) { if (sample == null) {
return; return;
} }
int slot = slot(x, z); int slot = slot(x, z);
@@ -76,7 +74,7 @@ final class SharedBoundsBiomes {
return; return;
} }
LONGS.setRelease(keys, slot, key(x, z)); LONGS.setRelease(keys, slot, key(x, z));
OBJECTS.setRelease(values, slot, biome); OBJECTS.setRelease(values, slot, sample);
INTS.setRelease(stamps, slot, stamp + 2); INTS.setRelease(stamps, slot, stamp + 2);
} }
@@ -49,6 +49,7 @@ import art.arcane.iris.generation.hydrology.cave.HydrologyCaveCell;
import art.arcane.iris.generation.hydrology.cave.HydrologyCaveStorage; import art.arcane.iris.generation.hydrology.cave.HydrologyCaveStorage;
import art.arcane.iris.generation.hydrology.raster.ChunkWaterRaster; import art.arcane.iris.generation.hydrology.raster.ChunkWaterRaster;
import art.arcane.iris.generation.chunk.ColumnExtent; import art.arcane.iris.generation.chunk.ColumnExtent;
import art.arcane.iris.generation.chunk.ColumnReadHunk;
import art.arcane.iris.generation.context.ChunkContext; import art.arcane.iris.generation.context.ChunkContext;
import art.arcane.iris.generation.context.IrisContext; import art.arcane.iris.generation.context.IrisContext;
import art.arcane.iris.generation.block.B; import art.arcane.iris.generation.block.B;
@@ -1086,19 +1087,21 @@ public class IrisCarveModifier extends EngineAssignedModifier<NativeBlockState>
int height = output.getHeight(); int height = output.getHeight();
ColumnExtent extent = output instanceof ColumnExtent columnExtent ? columnExtent : null; ColumnExtent extent = output instanceof ColumnExtent columnExtent ? columnExtent : null;
SolidMemo solids = new SolidMemo(); SolidMemo solids = new SolidMemo();
ColumnWindow column = new ColumnWindow(output, height);
for (int localX = 0; localX < width; localX++) { for (int localX = 0; localX < width; localX++) {
for (int localZ = 0; localZ < depth; localZ++) { for (int localZ = 0; localZ < depth; localZ++) {
int worldX = blockX + localX; int worldX = blockX + localX;
int worldZ = blockZ + localZ; int worldZ = blockZ + localZ;
int floor = -1; int floor = -1;
int top = extent == null ? height - 1 : extent.highestStoredY(localX, localZ); int top = extent == null ? height - 1 : extent.highestStoredY(localX, localZ);
column.begin(localX, localZ, top);
boolean previousKnown = false; boolean previousKnown = false;
boolean previousSolid = false; boolean previousSolid = false;
for (int y = 1; y < height; y++) { for (int y = 1; y < height; y++) {
if (y > top) { if (y > top) {
break; break;
} }
boolean solid = solids.test(output.getRaw(localX, y, localZ)); boolean solid = solids.test(column.get(y));
boolean decorated = false; boolean decorated = false;
if (solid) { if (solid) {
if (floor >= 0) { if (floor >= 0) {
@@ -1122,11 +1125,12 @@ public class IrisCarveModifier extends EngineAssignedModifier<NativeBlockState>
if (extent != null) { if (extent != null) {
top = extent.highestStoredY(localX, localZ); top = extent.highestStoredY(localX, localZ);
} }
column.begin(localX, localZ, top);
} }
floor = -1; floor = -1;
} }
} else if (floor < 0 && (previousKnown ? previousSolid } else if (floor < 0 && (previousKnown ? previousSolid
: solids.test(output.getRaw(localX, y - 1, localZ)))) { : solids.test(column.get(y - 1)))) {
floor = y; floor = y;
} }
previousSolid = solid; previousSolid = solid;
@@ -1472,6 +1476,43 @@ public class IrisCarveModifier extends EngineAssignedModifier<NativeBlockState>
private record CaveColumn(long key, IrisBiome surface, IrisBiome cave, int surfaceY, int minimumDepth) { private record CaveColumn(long key, IrisBiome surface, IrisBiome cave, int surfaceY, int minimumDepth) {
} }
private static final class ColumnWindow {
private static final int ROWS = 16;
private final Hunk<NativeBlockState> output;
private final NativeBlockState[] states;
private int x;
private int z;
private int limit;
private int from;
private int to;
private ColumnWindow(Hunk<NativeBlockState> output, int height) {
this.output = output;
this.states = new NativeBlockState[height];
}
private void begin(int localX, int localZ, int top) {
x = localX;
z = localZ;
limit = Math.min(top, states.length - 1);
from = 0;
to = -1;
}
private NativeBlockState get(int y) {
if (y < from || y > to) {
if (y > limit) {
return output.getRaw(x, y, z);
}
from = y;
to = Math.min(limit, y + ROWS - 1);
ColumnReadHunk.readColumn(output, x, z, from, to, states, from);
}
return states[y];
}
}
private static final class SolidMemo { private static final class SolidMemo {
private NativeBlockState state; private NativeBlockState state;
private boolean solid; private boolean solid;
@@ -19,6 +19,7 @@
package art.arcane.iris.generation.stage; package art.arcane.iris.generation.stage;
import art.arcane.iris.generation.chunk.ColumnExtent; import art.arcane.iris.generation.chunk.ColumnExtent;
import art.arcane.iris.generation.chunk.ColumnReadHunk;
import art.arcane.iris.generation.decoration.IrisSpeleothems; import art.arcane.iris.generation.decoration.IrisSpeleothems;
import art.arcane.iris.generation.runtime.Engine; import art.arcane.iris.generation.runtime.Engine;
import art.arcane.iris.generation.runtime.EngineAssignedModifier; import art.arcane.iris.generation.runtime.EngineAssignedModifier;
@@ -105,24 +106,29 @@ public class IrisPerfectionModifier extends EngineAssignedModifier<NativeBlockSt
StateFlags states = new StateFlags(); StateFlags states = new StateFlags();
NativeBlockState air = AIR.get(); NativeBlockState air = AIR.get();
NativeBlockState water = WATER.get(); NativeBlockState water = WATER.get();
NativeBlockState[] column = new NativeBlockState[output.getHeight()];
for (int x = firstSlice; x < lastSlice; x++) { for (int x = firstSlice; x < lastSlice; x++) {
for (int z = 0; z < 16; z++) { for (int z = 0; z < 16; z++) {
perfectColumn(output, x, z, surfaces, states, air, water); perfectColumn(output, x, z, surfaces, states, air, water, column);
} }
} }
} }
private static void perfectColumn(Hunk<NativeBlockState> output, int x, int z, IntArrayList surfaces, private static void perfectColumn(Hunk<NativeBlockState> output, int x, int z, IntArrayList surfaces,
StateFlags states, NativeBlockState air, NativeBlockState water) { StateFlags states, NativeBlockState air, NativeBlockState water,
NativeBlockState[] column) {
do { do {
surfaces.clear(); surfaces.clear();
int top = getHeight(output, x, z, states); int highest = Math.min(output.getHeight() - 1, ColumnExtent.highestStoredY(output, x, z));
ColumnReadHunk.readColumn(output, x, z, 0, Math.max(0, highest), column, 0);
int top = getHeight(column, highest, states);
boolean inside = true; boolean inside = true;
surfaces.add(top); surfaces.add(top);
NativeBlockState classified = null; NativeBlockState classified = null;
int classifiedFlags = 0; int classifiedFlags = 0;
boolean columnRead = true;
for (int y = top; y >= 0; y--) { for (int y = top; y >= 0; y--) {
NativeBlockState block = output.get(x, y, z); NativeBlockState block = columnRead ? column[y] : output.get(x, y, z);
int flags = 0; int flags = 0;
if (block != null) { if (block != null) {
if (block != classified) { if (block != classified) {
@@ -133,6 +139,7 @@ public class IrisPerfectionModifier extends EngineAssignedModifier<NativeBlockSt
if ((flags & SPIKE) != 0) { if ((flags & SPIKE) != 0) {
block = normalizeSpike(block, output, x, z, y, air, water); block = normalizeSpike(block, output, x, z, y, air, water);
flags = block == null ? 0 : states.of(block); flags = block == null ? 0 : states.of(block);
columnRead = false;
} }
} }
boolean now = (flags & SOLID) != 0; boolean now = (flags & SOLID) != 0;
@@ -233,11 +240,11 @@ public class IrisPerfectionModifier extends EngineAssignedModifier<NativeBlockSt
burst.complete(); burst.complete();
} }
private static int getHeight(Hunk<NativeBlockState> output, int x, int z, StateFlags states) { private static int getHeight(NativeBlockState[] column, int highest, StateFlags states) {
NativeBlockState classified = null; NativeBlockState classified = null;
int classifiedFlags = 0; int classifiedFlags = 0;
for (int i = Math.min(output.getHeight() - 1, ColumnExtent.highestStoredY(output, x, z)); i >= 0; i--) { for (int i = highest; i >= 0; i--) {
NativeBlockState state = output.get(x, i, z); NativeBlockState state = column[i];
if (state == null) { if (state == null) {
continue; continue;
} }
@@ -1,5 +1,6 @@
package art.arcane.iris.generation.stage; package art.arcane.iris.generation.stage;
import art.arcane.iris.generation.chunk.ColumnReadHunk;
import art.arcane.iris.generation.runtime.Engine; import art.arcane.iris.generation.runtime.Engine;
import art.arcane.iris.generation.runtime.EngineAssignedComponent; import art.arcane.iris.generation.runtime.EngineAssignedComponent;
import art.arcane.iris.generation.runtime.EngineStage; import art.arcane.iris.generation.runtime.EngineStage;
@@ -42,14 +43,14 @@ public final class IrisTransitionGeometryActuator extends EngineAssignedComponen
if (plan == null || !plan.hasTransitionAtChunk(x >> 4, z >> 4)) { if (plan == null || !plan.hasTransitionAtChunk(x >> 4, z >> 4)) {
return; return;
} }
GeometrySource source = new GeometrySource(blocks, biomes, getEngine().getMinHeight(), context); GeometrySource source = new GeometrySource(x, z, blocks, biomes, getEngine().getMinHeight(), context);
blendGeometry(x, z, blocks, context, source, plan); blendGeometry(x, z, blocks, context, source, plan);
} }
public static SavedTerrainChunk capture(int x, int z, Hunk<NativeBlockState> blocks, public static SavedTerrainChunk capture(int x, int z, Hunk<NativeBlockState> blocks,
Hunk<NativeBiome> biomes, int minimumY, Hunk<NativeBiome> biomes, int minimumY,
ChunkContext context, boolean boundaryOnly) throws IOException { ChunkContext context, boolean boundaryOnly) throws IOException {
GeometrySource source = new GeometrySource(blocks, biomes, minimumY, context); GeometrySource source = new GeometrySource(x, z, blocks, biomes, minimumY, context);
return boundaryOnly return boundaryOnly
? SavedTerrainChunk.captureBoundary(x >> 4, z >> 4, minimumY, blocks.getHeight(), "minecraft:noise", source) ? SavedTerrainChunk.captureBoundary(x >> 4, z >> 4, minimumY, blocks.getHeight(), "minecraft:noise", source)
: SavedTerrainChunk.capture(x >> 4, z >> 4, minimumY, blocks.getHeight(), "minecraft:noise", source); : SavedTerrainChunk.capture(x >> 4, z >> 4, minimumY, blocks.getHeight(), "minecraft:noise", source);
@@ -73,8 +74,10 @@ public final class IrisTransitionGeometryActuator extends EngineAssignedComponen
if (blended == current) { if (blended == current) {
continue; continue;
} }
double expectedFloor = context.getRoundedHeight(localX, localZ); int ground = context.getRoundedHeight(localX, localZ);
int floor = blended.surfaceOffsetNear(expectedFloor); BoundaryColumnGeometry.WaterCap cap = paintedCap(current, blended, ground,
waterHead(complex, x + localX, z + localZ, current.height()));
int floor = blended.surfaceOffsetNear(ground, cap);
List<BoundaryColumnGeometry.Voxel> voxels = blended.voxels(); List<BoundaryColumnGeometry.Voxel> voxels = blended.voxels();
for (int offset = 0; offset < voxels.size(); offset++) { for (int offset = 0; offset < voxels.size(); offset++) {
BoundaryColumnGeometry.Voxel voxel = voxels.get(offset); BoundaryColumnGeometry.Voxel voxel = voxels.get(offset);
@@ -94,6 +97,9 @@ public final class IrisTransitionGeometryActuator extends EngineAssignedComponen
&& voxels.get(fluidHeight + 1).phase() == BoundaryColumnGeometry.Phase.FLUID) { && voxels.get(fluidHeight + 1).phase() == BoundaryColumnGeometry.Phase.FLUID) {
fluidHeight++; fluidHeight++;
} }
if (blended.capsWater(cap, floor)) {
fluidHeight = cap.top();
}
if (fluidHeight == floor) { if (fluidHeight == floor) {
fluidHeight = -1; fluidHeight = -1;
} }
@@ -104,17 +110,43 @@ public final class IrisTransitionGeometryActuator extends EngineAssignedComponen
} }
} }
private static BoundaryColumnGeometry.WaterCap paintedCap(BoundaryColumnGeometry painted, BoundaryColumnGeometry blended,
int ground, int waterHead) {
BoundaryColumnGeometry.WaterCap cap = painted.waterCap(ground, waterHead);
if (!cap.present()) {
return cap;
}
int top = Math.min(cap.top() + 1, painted.height() - 1);
for (int offset = cap.bottom(); offset <= top; offset++) {
int y = painted.minimumY() + offset;
if (!painted.voxelAt(y).equals(blended.voxelAt(y))) {
return BoundaryColumnGeometry.WaterCap.none();
}
}
return cap;
}
private static int waterHead(IrisComplex complex, int blockX, int blockZ, int height) {
return Math.clamp(Math.round(complex.getWaterHeadStream().getDouble(blockX, blockZ)), -1, height - 1);
}
private static final class GeometrySource implements SavedTerrainChunk.VoxelSource { private static final class GeometrySource implements SavedTerrainChunk.VoxelSource {
private final int originX;
private final int originZ;
private final Hunk<NativeBlockState> blocks; private final Hunk<NativeBlockState> blocks;
private final Hunk<NativeBiome> biomes; private final Hunk<NativeBiome> biomes;
private final int minimumY; private final int minimumY;
private final ChunkContext context; private final ChunkContext context;
private final Map<NativeBlockState, BoundaryColumnGeometry.Voxel> voxels = new IdentityHashMap<>(); private final Map<NativeBlockState, BoundaryColumnGeometry.Voxel> voxels = new IdentityHashMap<>();
private NativeBlockState air; private NativeBlockState air;
private NativeBlockState[] column = new NativeBlockState[0];
private NativeBlockState lastState; private NativeBlockState lastState;
private BoundaryColumnGeometry.Voxel lastVoxel; private BoundaryColumnGeometry.Voxel lastVoxel;
private GeometrySource(Hunk<NativeBlockState> blocks, Hunk<NativeBiome> biomes, int minimumY, ChunkContext context) { private GeometrySource(int originX, int originZ, Hunk<NativeBlockState> blocks, Hunk<NativeBiome> biomes,
int minimumY, ChunkContext context) {
this.originX = originX;
this.originZ = originZ;
this.blocks = blocks; this.blocks = blocks;
this.biomes = biomes; this.biomes = biomes;
this.minimumY = minimumY; this.minimumY = minimumY;
@@ -132,10 +164,14 @@ public final class IrisTransitionGeometryActuator extends EngineAssignedComponen
return; return;
} }
int base = minimumY - this.minimumY; int base = minimumY - this.minimumY;
NativeBlockState runState = stateAt(localX, base, localZ); if (column.length < height) {
column = new NativeBlockState[height];
}
ColumnReadHunk.readColumn(blocks, localX, localZ, base, base + height - 1, column, 0);
NativeBlockState runState = orAir(column[0]);
int start = 0; int start = 0;
for (int offset = 1; offset < height; offset++) { for (int offset = 1; offset < height; offset++) {
NativeBlockState state = stateAt(localX, base + offset, localZ); NativeBlockState state = orAir(column[offset]);
if (state != runState) { if (state != runState) {
runs.add(start, offset, voxelOf(runState)); runs.add(start, offset, voxelOf(runState));
start = offset; start = offset;
@@ -146,7 +182,10 @@ public final class IrisTransitionGeometryActuator extends EngineAssignedComponen
} }
private NativeBlockState stateAt(int localX, int offset, int localZ) { private NativeBlockState stateAt(int localX, int offset, int localZ) {
NativeBlockState state = blocks.getRaw(localX, offset, localZ); return orAir(blocks.getRaw(localX, offset, localZ));
}
private NativeBlockState orAir(NativeBlockState state) {
if (state != null) { if (state != null) {
return state; return state;
} }
@@ -169,6 +208,11 @@ public final class IrisTransitionGeometryActuator extends EngineAssignedComponen
return OptionalInt.of(Math.max(0, Math.min(blocks.getHeight() - 1, context.getRoundedHeight(localX, localZ)))); return OptionalInt.of(Math.max(0, Math.min(blocks.getHeight() - 1, context.getRoundedHeight(localX, localZ))));
} }
@Override
public OptionalInt waterSurface(int localX, int localZ) {
return OptionalInt.of(waterHead(context.getComplex(), originX + localX, originZ + localZ, blocks.getHeight()));
}
@Override @Override
public String biome(int localX, int worldY, int localZ) throws IOException { public String biome(int localX, int worldY, int localZ) throws IOException {
NativeBiome biome = biomes.getRaw(localX, worldY - minimumY, localZ); NativeBiome biome = biomes.getRaw(localX, worldY - minimumY, localZ);
@@ -39,6 +39,13 @@ public final class BakedStream<T> extends BasicStream<T> implements ChunkFillabl
return value; return value;
} }
@SuppressWarnings("unchecked")
public T baked(int blockX, int blockZ) {
int index = ((blockZ & 15) << 4) | (blockX & 15);
ChunkPlanes.Plane values = cache.planeIfPresent(plane, blockX >> 4, blockZ >> 4);
return values != null && values.ready(index) ? (T) values.objectAt(index) : null;
}
@Override @Override
public T get(double x, double y, double z) { public T get(double x, double y, double z) {
return scalar.get(x, y, z); return scalar.get(x, y, z);
@@ -0,0 +1,74 @@
/*
* Iris is a World Generator for Minecraft Bukkit Servers
* Copyright (c) 2026 Arcane Arts (Volmit Software)
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
package art.arcane.iris.generation.validation;
import art.arcane.iris.generation.concurrent.MultiBurst;
import art.arcane.iris.spi.IrisLogging;
import java.util.List;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.Semaphore;
import java.util.concurrent.atomic.AtomicInteger;
public final class ChunkSpiralScan {
private ChunkSpiralScan() {
}
public static int scan(List<int[]> targets, int threads, ChunkVisitor visitor, Listener listener) throws InterruptedException {
Semaphore inFlight = new Semaphore(Math.max(1, threads));
CountDownLatch done = new CountDownLatch(targets.size());
AtomicInteger failures = new AtomicInteger();
for (int[] target : targets) {
int chunkX = target[0];
int chunkZ = target[1];
inFlight.acquire();
MultiBurst.burst.lazy(() -> {
boolean ok = false;
try {
visitor.visit(chunkX, chunkZ);
ok = true;
} catch (Throwable e) {
failures.incrementAndGet();
IrisLogging.reportError(e);
listener.chunkFailed(chunkX, chunkZ, e);
} finally {
listener.chunkDone(chunkX, chunkZ, ok);
inFlight.release();
done.countDown();
}
});
}
done.await();
return failures.get();
}
@FunctionalInterface
public interface ChunkVisitor {
void visit(int chunkX, int chunkZ) throws Exception;
}
public interface Listener {
void chunkDone(int chunkX, int chunkZ, boolean ok);
void chunkFailed(int chunkX, int chunkZ, Throwable error);
}
}
@@ -26,7 +26,6 @@ import art.arcane.iris.spi.IrisLogging;
import art.arcane.volmlib.nativelib.terrain.NativeBiome; import art.arcane.volmlib.nativelib.terrain.NativeBiome;
import art.arcane.volmlib.nativelib.terrain.NativeBlockState; import art.arcane.volmlib.nativelib.terrain.NativeBlockState;
import art.arcane.volmlib.util.math.ChunkSpiral; import art.arcane.volmlib.util.math.ChunkSpiral;
import art.arcane.iris.generation.concurrent.MultiBurst;
import art.arcane.volmlib.util.localization.MessageArgument; import art.arcane.volmlib.util.localization.MessageArgument;
import art.arcane.volmlib.util.mantle.runtime.Mantle; import art.arcane.volmlib.util.mantle.runtime.Mantle;
@@ -44,8 +43,6 @@ import java.util.List;
import java.util.Map; import java.util.Map;
import java.util.TreeMap; import java.util.TreeMap;
import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.Semaphore;
import java.util.concurrent.atomic.AtomicInteger; import java.util.concurrent.atomic.AtomicInteger;
public final class GoldenHashEngine { public final class GoldenHashEngine {
@@ -210,40 +207,18 @@ public final class GoldenHashEngine {
private Map<Long, String> scan(List<int[]> targets) throws InterruptedException { private Map<Long, String> scan(List<int[]> targets) throws InterruptedException {
Map<Long, String> lines = new ConcurrentHashMap<>(); Map<Long, String> lines = new ConcurrentHashMap<>();
Semaphore inFlight = new Semaphore(threads); ChunkSpiralScan.scan(targets, threads, (int chunkX, int chunkZ) -> hash(lines, chunkX, chunkZ), new ScanProgress(targets.size()));
CountDownLatch done = new CountDownLatch(targets.size());
AtomicInteger completed = new AtomicInteger();
int total = targets.size();
for (int[] target : targets) {
int chunkX = target[0];
int chunkZ = target[1];
inFlight.acquire();
MultiBurst.burst.lazy(() -> {
boolean ok = false;
try {
ChunkSnapshot snapshot = source.generate(chunkX, chunkZ);
lines.put(chunkKey(chunkX, chunkZ), hashChunk(chunkX, chunkZ, snapshot));
if (request.deep()) {
writeDeepDump(chunkX, chunkZ, snapshot);
}
ok = true;
} catch (Throwable e) {
IrisLogging.reportError(e);
progress.chunkFailed(chunkX, chunkZ, e);
} finally {
progress.chunkDone(chunkX, chunkZ, ok, completed.incrementAndGet(), total);
inFlight.release();
done.countDown();
}
});
}
done.await();
return lines; return lines;
} }
private void hash(Map<Long, String> lines, int chunkX, int chunkZ) throws Exception {
ChunkSnapshot snapshot = source.generate(chunkX, chunkZ);
lines.put(chunkKey(chunkX, chunkZ), hashChunk(chunkX, chunkZ, snapshot));
if (request.deep()) {
writeDeepDump(chunkX, chunkZ, snapshot);
}
}
private String hashChunk(int chunkX, int chunkZ, ChunkSnapshot snapshot) { private String hashChunk(int chunkX, int chunkZ, ChunkSnapshot snapshot) {
MessageDigest blockDigest = sha256(); MessageDigest blockDigest = sha256();
MessageDigest biomeDigest = sha256(); MessageDigest biomeDigest = sha256();
@@ -562,4 +537,23 @@ public final class GoldenHashEngine {
throw new IllegalStateException(e); throw new IllegalStateException(e);
} }
} }
private final class ScanProgress implements ChunkSpiralScan.Listener {
private final AtomicInteger completed = new AtomicInteger();
private final int total;
private ScanProgress(int total) {
this.total = total;
}
@Override
public void chunkDone(int chunkX, int chunkZ, boolean ok) {
progress.chunkDone(chunkX, chunkZ, ok, completed.incrementAndGet(), total);
}
@Override
public void chunkFailed(int chunkX, int chunkZ, Throwable error) {
progress.chunkFailed(chunkX, chunkZ, error);
}
}
} }
@@ -33,6 +33,74 @@ public final class DirectorCommandMessages {
"iris.director.commanddatapack.param.datapack_id_folder_name_shown_by_iris_datapack_list", "iris.director.commanddatapack.param.datapack_id_folder_name_shown_by_iris_datapack_list",
"The datapack id (folder name) shown by /iris datapack list" "The datapack id (folder name) shown by /iris datapack list"
); );
public static final TextKey COMMAND_DATAPACK_EXPORT_DESCRIPTION = TextKey.of(
"iris.director.commanddatapack.director.export_dimension_vanilla_worldgen_datapack",
"Export a dimension as a vanilla worldgen datapack"
);
public static final TextKey COMMAND_DATAPACK_EXPORT_PARAM_DIMENSION = TextKey.of(
"iris.director.commanddatapack.param.export_dimension",
"The dimension to export"
);
public static final TextKey COMMAND_DATAPACK_EXPORT_PARAM_VERSION = TextKey.of(
"iris.director.commanddatapack.param.export_version",
"Target Minecraft version, 26.3 or later (blank uses the latest)"
);
public static final TextKey COMMAND_DATAPACK_EXPORT_PARAM_MODE = TextKey.of(
"iris.director.commanddatapack.param.export_mode",
"Output mode: preset, dimension, or overworld"
);
public static final TextKey COMMAND_DATAPACK_EXPORT_PARAM_NAMESPACE = TextKey.of(
"iris.director.commanddatapack.param.export_namespace",
"Namespace for the exported resources"
);
public static final TextKey COMMAND_DATAPACK_EXPORT_PARAM_OUTPUT = TextKey.of(
"iris.director.commanddatapack.param.export_output",
"Output name in the datapack exports folder (blank uses the dimension and version)"
);
public static final TextKey COMMAND_DATAPACK_EXPORT_PARAM_FORMAT = TextKey.of(
"iris.director.commanddatapack.param.export_format",
"zip or directory"
);
public static final TextKey COMMAND_DATAPACK_EXPORT_PARAM_OBJECTS = TextKey.of(
"iris.director.commanddatapack.param.export_objects",
"Include objects as structure templates"
);
public static final TextKey COMMAND_DATAPACK_EXPORT_PARAM_REPLACE = TextKey.of(
"iris.director.commanddatapack.param.export_replace",
"Replace an existing export with the same name"
);
public static final TextKey COMMAND_DATAPACK_EXPORT_STARTED = TextKey.of(
"iris.director.commanddatapack.export.started",
"Exporting {dimension} as a datapack for Minecraft {version}..."
);
public static final TextKey COMMAND_DATAPACK_EXPORT_STAGE = TextKey.of(
"iris.director.commanddatapack.export.stage",
"Datapack export: {stage}"
);
public static final TextKey COMMAND_DATAPACK_EXPORT_RESOURCES = TextKey.of(
"iris.director.commanddatapack.export.resources",
"Datapack export: {done}/{total} resources"
);
public static final TextKey COMMAND_DATAPACK_EXPORT_DIAGNOSTIC = TextKey.of(
"iris.director.commanddatapack.export.diagnostic",
"{key}: {message}"
);
public static final TextKey COMMAND_DATAPACK_EXPORT_EXPORTED = TextKey.of(
"iris.director.commanddatapack.export.exported",
"Datapack exported: {path}"
);
public static final TextKey COMMAND_DATAPACK_EXPORT_REJECTED = TextKey.of(
"iris.director.commanddatapack.export.rejected",
"Datapack export rejected: {reason}"
);
public static final TextKey COMMAND_DATAPACK_EXPORT_FAILED = TextKey.of(
"iris.director.commanddatapack.export.failed",
"Datapack export failed: {reason}"
);
public static final TextKey COMMAND_DATAPACK_EXPORT_INVALID = TextKey.of(
"iris.director.commanddatapack.export.invalid",
"Invalid datapack export arguments: {reason}"
);
public static final TextKey COMMAND_DEVELOPER_DIRECTOR_IRIS_WORLD_MANAGER = TextKey.of( public static final TextKey COMMAND_DEVELOPER_DIRECTOR_IRIS_WORLD_MANAGER = TextKey.of(
"iris.director.commanddeveloper.director.iris_world_manager", "iris.director.commanddeveloper.director.iris_world_manager",
"Iris World Manager" "Iris World Manager"
@@ -888,6 +956,23 @@ public final class DirectorCommandMessages {
COMMAND_DATAPACK_DIRECTOR_LIST_CONFIGURED_DATAPACK_IMPORTS_THEIR_INSTALLED_VERSIONS, COMMAND_DATAPACK_DIRECTOR_LIST_CONFIGURED_DATAPACK_IMPORTS_THEIR_INSTALLED_VERSIONS,
COMMAND_DATAPACK_DIRECTOR_REMOVE_INSTALLED_DATAPACK_BY_ID_ALSO_DELETE_ITS_URL_FROM_DATAPACKIMPORTS_KEEP, COMMAND_DATAPACK_DIRECTOR_REMOVE_INSTALLED_DATAPACK_BY_ID_ALSO_DELETE_ITS_URL_FROM_DATAPACKIMPORTS_KEEP,
COMMAND_DATAPACK_PARAM_DATAPACK_ID_FOLDER_NAME_SHOWN_BY_IRIS_DATAPACK_LIST, COMMAND_DATAPACK_PARAM_DATAPACK_ID_FOLDER_NAME_SHOWN_BY_IRIS_DATAPACK_LIST,
COMMAND_DATAPACK_EXPORT_DESCRIPTION,
COMMAND_DATAPACK_EXPORT_PARAM_DIMENSION,
COMMAND_DATAPACK_EXPORT_PARAM_VERSION,
COMMAND_DATAPACK_EXPORT_PARAM_MODE,
COMMAND_DATAPACK_EXPORT_PARAM_NAMESPACE,
COMMAND_DATAPACK_EXPORT_PARAM_OUTPUT,
COMMAND_DATAPACK_EXPORT_PARAM_FORMAT,
COMMAND_DATAPACK_EXPORT_PARAM_OBJECTS,
COMMAND_DATAPACK_EXPORT_PARAM_REPLACE,
COMMAND_DATAPACK_EXPORT_STARTED,
COMMAND_DATAPACK_EXPORT_STAGE,
COMMAND_DATAPACK_EXPORT_RESOURCES,
COMMAND_DATAPACK_EXPORT_DIAGNOSTIC,
COMMAND_DATAPACK_EXPORT_EXPORTED,
COMMAND_DATAPACK_EXPORT_REJECTED,
COMMAND_DATAPACK_EXPORT_FAILED,
COMMAND_DATAPACK_EXPORT_INVALID,
COMMAND_DEVELOPER_DIRECTOR_IRIS_WORLD_MANAGER, COMMAND_DEVELOPER_DIRECTOR_IRIS_WORLD_MANAGER,
COMMAND_DEVELOPER_DIRECTOR_GET_LOADED_TECTONICPLATES_COUNT, COMMAND_DEVELOPER_DIRECTOR_GET_LOADED_TECTONICPLATES_COUNT,
COMMAND_DEVELOPER_DIRECTOR_HASH_GENERATED_BLOCK_OUTPUT_FIXED_AREA_DETERMINISM_IDENTITY_TESTING, COMMAND_DEVELOPER_DIRECTOR_HASH_GENERATED_BLOCK_OUTPUT_FIXED_AREA_DETERMINISM_IDENTITY_TESTING,
@@ -13,20 +13,24 @@ import art.arcane.iris.pack.datapack.v263.DataFixerV263;
//https://minecraft.wiki/w/Pack_format //https://minecraft.wiki/w/Pack_format
@Getter @Getter
public enum DataVersion { public enum DataVersion {
UNSUPPORTED("0.0.0", 0, () -> null), UNSUPPORTED("0.0.0", 0, 0, 0, () -> null),
V1_21_1("1.21.1", 48, DataFixerV1206::new), V1_21_1("1.21.1", 48, 0, 3955, DataFixerV1206::new),
V26_1_2("26.1.2", 101, DataFixerV1217::new), V26_1_2("26.1.2", 101, 1, 4790, DataFixerV1217::new),
V26_2("26.2", 107, DataFixerV1217::new), V26_2("26.2", 107, 1, 4903, DataFixerV1217::new),
V26_3("26.3", 121, DataFixerV263::new); V26_3("26.3", 121, 0, 5023, DataFixerV263::new);
private static final KMap<DataVersion, IDataFixer> cache = new KMap<>(); private static final KMap<DataVersion, IDataFixer> cache = new KMap<>();
@Getter(AccessLevel.NONE) @Getter(AccessLevel.NONE)
private final Supplier<IDataFixer> constructor; private final Supplier<IDataFixer> constructor;
private final String version; private final String version;
private final int packFormat; private final int packFormat;
private final int packFormatMinor;
private final int worldDataVersion;
DataVersion(String version, int packFormat, Supplier<IDataFixer> constructor) { DataVersion(String version, int packFormat, int packFormatMinor, int worldDataVersion, Supplier<IDataFixer> constructor) {
this.constructor = constructor; this.constructor = constructor;
this.packFormat = packFormat; this.packFormat = packFormat;
this.packFormatMinor = packFormatMinor;
this.worldDataVersion = worldDataVersion;
this.version = version; this.version = version;
} }
@@ -0,0 +1,99 @@
package art.arcane.iris.pack.datapack;
import art.arcane.iris.pack.datapack.DatapackIngestService.Entry;
import art.arcane.iris.pack.datapack.ModrinthResolver.ResolvedDatapack;
import art.arcane.iris.spi.IrisLogging;
import art.arcane.iris.spi.IrisPlatform;
import art.arcane.iris.spi.IrisPlatforms;
import art.arcane.volmlib.util.io.IO;
import java.io.File;
import java.io.IOException;
import java.net.URI;
import java.nio.file.LinkOption;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Objects;
public final class DatapackExportArchives {
private static final String DIRECT_VERSION_PREFIX = "direct-";
private DatapackExportArchives() {
}
public static Path resolve(String source, DataVersion target) throws IOException {
String normalized = DatapackImportSources.normalizeConfiguredSource(Objects.requireNonNull(source, "datapack import source"));
URI uri = DatapackStartupValidation.parseSourceUri(normalized);
if (uri == null) {
throw new IOException("Empty datapack import source");
}
if ("file".equalsIgnoreCase(uri.getScheme())) {
return DatapackArchive.requireLocalDatapackPath(uri);
}
IrisPlatform platform = IrisPlatforms.get();
File root = platform.dataFolder("datapacks");
File cacheDir = new File(root, "cache");
boolean running = DataVersion.forMinecraftVersion(platform.minecraftVersion()) == target;
DatapackTransactions.TRANSACTION_LOCK.lock();
try {
Entry entry = manifestEntry(root, normalized);
Path cached = ingestCache(cacheDir, entry, running);
if (cached != null) {
return cached;
}
return download(cacheDir, normalized, entry, target);
} finally {
DatapackTransactions.TRANSACTION_LOCK.unlock();
}
}
private static Entry manifestEntry(File root, String source) {
try {
return DatapackManifestStore.readCommittedManifest(root).find(source);
} catch (IOException exception) {
IrisLogging.reportError("The datapack manifest under " + root.getPath() + " could not be read; '" + source
+ "' is resolved again for the export", exception);
return null;
}
}
private static Path ingestCache(File cacheDir, Entry entry, boolean running) throws IOException {
if (entry == null || entry.id == null || entry.sha1 == null) {
return null;
}
boolean direct = entry.versionId != null && entry.versionId.startsWith(DIRECT_VERSION_PREFIX);
if (!direct && !running) {
return null;
}
File archive = new File(cacheDir, entry.id + "-" + DatapackSupport.safeFile(entry.versionId) + ".zip");
return matches(archive, entry.sha1) ? archive.toPath() : null;
}
private static Path download(File cacheDir, String source, Entry entry, DataVersion target) throws IOException {
ResolvedDatapack resolved = ModrinthResolver.resolve(source, target.getVersion());
String id = entry == null || entry.id == null ? DatapackArchive.deriveId(resolved) : entry.id;
File archive = new File(cacheDir, id + "-" + DatapackSupport.safeFile(resolved.getVersionId()) + ".zip");
String expected = resolved.getSha1();
if (expected != null && !expected.isBlank() && matches(archive, expected)) {
return archive.toPath();
}
DatapackSupport.ensureScratchDirectory(cacheDir, "datapack download cache");
DatapackArchive.download(resolved.getDownloadUrl(), archive, null, null);
if (expected != null && !expected.isBlank()) {
String checksum = DatapackArchive.sha1(archive);
if (!expected.equalsIgnoreCase(checksum)) {
IO.delete(archive);
throw new IOException("Checksum mismatch for " + id + " (expected " + expected + ", got " + checksum + ")");
}
}
return archive.toPath();
}
private static boolean matches(File archive, String sha1) throws IOException {
Path path = archive.toPath();
if (Files.isSymbolicLink(path) || !Files.isRegularFile(path, LinkOption.NOFOLLOW_LINKS)) {
return false;
}
return sha1.equalsIgnoreCase(DatapackArchive.sha1(archive));
}
}
@@ -1,4 +1,4 @@
package art.arcane.iris.structure.export; package art.arcane.iris.pack.datapack.export;
import java.io.IOException; import java.io.IOException;
import java.nio.file.AtomicMoveNotSupportedException; import java.nio.file.AtomicMoveNotSupportedException;
@@ -14,25 +14,26 @@ import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream; import java.util.zip.ZipOutputStream;
import java.util.stream.Stream; import java.util.stream.Stream;
final class AtomicDatapackPublisher { public final class AtomicDatapackPublisher {
Publication publish( public Publication publish(
Map<String, byte[]> resources, Map<String, byte[]> files,
Path output, Path destination,
VanillaJigsawExportFormat format, DatapackExportFormat format,
boolean replaceExisting boolean replace,
String stagingPrefix
) throws IOException { ) throws IOException {
Path parent = output.getParent(); Path parent = destination.getParent();
if (parent == null) { if (parent == null) {
throw new IOException("Export output has no parent directory: " + output); throw new IOException("Export output has no parent directory: " + destination);
} }
Files.createDirectories(parent); Files.createDirectories(parent);
Path staging = Files.createTempDirectory(parent, ".iris-jigsaw-export-"); Path staging = Files.createTempDirectory(parent, stagingPrefix);
List<String> cleanupWarnings = new ArrayList<>(); List<String> cleanupWarnings = new ArrayList<>();
try { try {
Path artifact = format == VanillaJigsawExportFormat.DIRECTORY Path artifact = format == DatapackExportFormat.DIRECTORY
? createDirectoryArtifact(staging, resources) ? createDirectoryArtifact(staging, files)
: createZipArtifact(staging, resources); : createZipArtifact(staging, files);
replaceAtomically(artifact, output, replaceExisting, cleanupWarnings); replaceAtomically(artifact, destination, replace, cleanupWarnings);
} finally { } finally {
try { try {
deleteTree(staging); deleteTree(staging);
@@ -152,8 +153,8 @@ final class AtomicDatapackPublisher {
} }
} }
record Publication(List<String> cleanupWarnings) { public record Publication(List<String> cleanupWarnings) {
Publication { public Publication {
cleanupWarnings = List.copyOf(cleanupWarnings); cleanupWarnings = List.copyOf(cleanupWarnings);
} }
} }
@@ -0,0 +1,22 @@
package art.arcane.iris.pack.datapack.export;
public record ContentTrim(int filesBefore, long bytesBefore, int filesAfter, long bytesAfter) {
public static final ContentTrim NONE = new ContentTrim(0, 0L, 0, 0L);
public ContentTrim {
if (filesBefore < 0 || bytesBefore < 0L || filesAfter < 0 || bytesAfter < 0L) {
throw new IllegalArgumentException("Content trim counts cannot be negative");
}
if (filesAfter > filesBefore || bytesAfter > bytesBefore) {
throw new IllegalArgumentException("A content trim cannot add files or bytes");
}
}
public int filesRemoved() {
return filesBefore - filesAfter;
}
public long bytesRemoved() {
return bytesBefore - bytesAfter;
}
}
Loaded 100 of 498 files, more files were not shown because too many files have changed in this diff. Show more