Audio files referenced directly in GUI/simulation JS strings
(e.g. g_FlareSound, soundfile entries, Ambient tracks) were not
tracked. Add a js_audio_regex scan alongside the existing sprite
scan for both gui/ and simulation/ JS files.
Several categories of session UI textures were falsely reported as
unused because checkrefs only parsed XML/JSON data files and missed
texture references buried in JavaScript runtime logic.
Entity templates:
- Parse <Identity><Rank> to track session/icons/ranks/{rank}.png
- Parse <Minimap><Icon> to track session/icons/minimap/{icon}.png
Data-driven JSON files:
- Add add_resources(): scan simulation/data/resources/*.json for
resource 'code' fields -> session/icons/resources/{code}.png
- Add add_status_effects(): scan simulation/data/status_effects/*.json
for 'icon' fields -> session/icons/status_effects/{icon}.png
Technology files:
- In add_techs(): detect phase_*.json base techs (e.g. phase_city.json)
and add session/panel_phase_emblems_{phase}.png as a dep
JS scanning (extended):
- Extend the GUI JS literal-path regex to also recognise:
- 'texture:' sprite modifier prefix
- full 'art/textures/ui/...' literal paths
- bare 'session/...' paths used as runtime sprite components
- Also scan simulation/components, simulation/helpers and globalscripts
JS files (registered as roots) with the same regex so textures
referenced from simulation code are tracked
- Add _add_unit_action_icon_deps(): parse unit_actions.js for
'icon': 'X.png' properties -> session/icons/X
- Add _add_status_bar_icon_deps(): parse StatusBars.js AddBar() calls
for bar type strings -> session/icons/{type}_bg/fg.png
- Add _add_stance_icon_deps(): parse g_Stances in UnitAI.js for
selectable stance keys -> session/icons/stances/{stance}.png
Two sources of false-positive unused-file warnings:
1. Literal stretched:/cropped: sprite paths in GUI JS files (e.g.
bell_level0.png, cancel.png, minimap flare textures) were never
tracked because JS is not parsed for texture deps. Add a regex scan
of gui/*.js for complete literal sprite paths ending in a known
texture extension, filtering out dynamic string concatenations
(+ / ${...}).
2. Per-civ background circle textures (bkg/background_circle_<code>.png)
are constructed at runtime in MiniMapPanel.js via a template literal
using the player civ code. Add a dep from each civ JSON to its
corresponding background circle texture.
Icons defined in setup XML files (e.g. lobby/icons/public.xml) have
sprite attributes that reference UI textures, but the setup parser only
handled <scrollbar>, <tooltip> and <color> tags. This caused textures
like lobby/public.png to be falsely reported as unused by -u.
Some engine assets are loaded directly by C++ renderer code with no
corresponding reference in any XML template, actor, or data file that
checkrefs.py can statically trace. Files in mod layers that override
these paths were incorrectly reported as unused.
Introduce source/tools/entity/cpp_hardcoded_assets.json to maintain
the list of such VFS paths alongside their C++ source location and a
description. checkrefs.py reads this JSON via add_cpp_hardcoded_assets()
and marks every matching file in all loaded mod layers as a known-used
root, so mod overrides are no longer flagged.
Initial entries:
art/textures/terrain/types/water/coastalWave.png
art/textures/terrain/types/water/foam.png
Both are loaded unconditionally by WaterManager::LoadWaterTextures().
Fixes 2 false-positive unused-file warnings for mod/ water textures.
TerrainTextureManager::LoadAlphaMap() hardcodes 14 PNG filenames and
loads them from art/textures/terrain/alphamaps/<type>/ where <type>
comes from the terrain XML <alphamap> element (defaulting to
"standard"). checkrefs add_terrains() parsed <textures> and
<material> but never read <alphamap> and had no knowledge of the
filenames loaded by C++.
Changes:
- Read <alphamap> from each terrain XML (default "standard") and
add all 14 hardcoded PNG filenames as deps of the terrain file.
- Always register the "standard" set as roots since C++ loads it
unconditionally as a fallback, even when no terrain specifies it.
The filenames are kept in sync with the fnames[] array in
source/graphics/TerrainTextureManager.cpp.
Fixes 14 false-positive unused-file warnings for the standard/
alphamap set.
The entity parser handled LineTexture/LineTextureMask under
Heal/RangeOverlay and MainTexture/MainTextureMask under
Selectable/Overlay/Texture, but had no handling for the
Selectable/Overlay/Outline/{LineTexture,LineTextureMask} path
used by the outline selection ring renderer.
These textures are referenced in e.g. mixins/trading_post.xml and
any other template that sets a custom outline selection style.
Fixes 2 false-positive unused-file warnings:
art/textures/selection/outline_border_02.png
art/textures/selection/outline_border_02_mask.png
The sprites XML parser iterated sprite.attrib.items() — the attributes
of the <sprite> tag itself — and checked if el == "image". Since
<image texture="..."> elements are children of <sprite>, not its
attributes, this check could never be true. No texture dependency was
ever extracted from any sprites definition file.
Fix by iterating sprite.findall("image") and sprite.findall("effect")
to correctly traverse child elements, and read attributes from those
elements directly.
Fixes 99 false-positive unused-file warnings across global/border/,
global/button/, global/modern/, global/watermark/, loading/,
pregame/, session/, and tipdisplay/ UI texture directories.
checkrefs.py only traced music files referenced in civ JSON Music[]
arrays. Two additional sources were missed:
- gui/common/music.js DEFAULT_*_TRACKS prototype arrays define fallback
playlists used when a civ has no music entries. All .ogg filenames
found in that file are now added as deps of the music.js root.
- simulation/data/cheats/*.json files can reference audio files via
Data.DefaultParameter (e.g. retroMe.json -> 0_8Bit_Bonus_Track.ogg).
These are now tracked as deps of each cheat root.
Fixes 10 false-positive unused-file warnings.
With v3.12.0 shfmt changed the behavior of the `--simplify` option to
ignore any EditorConfig when it's provided. Therefore, this option is
now moved to the EditorConfig as suggested by the shfmt author. See
https://github.com/mvdan/sh/issues/1173 for details.
This refactors validate_dae.py to utilize multiple CPU cores. This makes
it significantly faster. It also improves code quality and adds commands
line options.
Instead of listing files recursively for each directory manually, this
uses the built-in glob functionality of Python's pathlib.Path. As
part of this the found paths are returned as generator instead of a
list. Together this results in easier to read code and better
performance.
A typo in the assignment of the error state lead to checkrefs.py
existing with return code 0, while it should've returned 1 in some
cases. This commit fixes that.
As the fonts specified in the GUI templates don't exist as files on disk
anymore after 734386ce9f we can't check if they're specified correctly
the same way as before anymore. Instead, this commit adds validation
of font names to the GUI Relax NG schema and adds a check whether all
fonts referenced in default.cfg exist in the proper location.
A multi player loading screen will have possibility of showing single
and multi player tips while multi player tips are in favor.
Tips are now separated by categories. During loading those category
have an occurrence probability, where we are choosing a random category
based on chance and not by complete randomness. From there we just
picking a random tip from that category.
The most notable changes are:
- improved performance thanks to using `lxml` instead of `xmllint` for
validating the templates
- improved performance by parallelizing the validation across multiple
threads
- more meaningful output for detected validation errors
- easier to call from other scripts (like checkrefs.py)
The xmlvalidator logic is only used by the checkrefs.py script, so this
moves it to the same directory to have it co-located and avoid having to
modify sys.path to import it.
Add checks for sprites, styles, scrollbars, tooltips, and colors.
And too for entire objects referenced by the "use_object" attribute.
(even though that is currently unused)
Enable the Tips and Tricks page to show multiple images for each tip.
Add a small GUI panel below the image for switching between them.
Only show it when the current tip has more than one images.
Introduce tipfiles.json to assign images to the text files.
(And stop of relying on file naming)
Add a tip about wonders, and make some slight changes to other texts.
Improve existing tip images and add a bunch of new ones.
- Small popup page accessible from the main menu via Learn to Play.
- Lets the player read through the tips with 'Previous' and 'Next' buttons.
- Tips continue to be shown on the loading screen.
-> but there without the scrolling ability.
- Added two new hotkeys for quicker tip changing (item.prev and item.next)
-> set to the left and right arrows respectively by default.
- Responsible scripts are placed in gui/reference/tips/.
- The tip text files have been moved to gui/reference/tips/texts/.
- Tip image files have been moved to art/textures/ui/tips/.
- Added a series of new sprites (textures in art/textures/ui/tipdisplay/)
-> comprises a title decoration, a bullet point sprite, and a new button
style.
This ensures the same Python target version used for `ruff format`
is used for `ruff check` as well. It also allows ruff, even if it's not
run through pre-commit, to use the correct target Python version.
This enables some ruff rules for docstrings and comments. The idea is to
not enforce the presence of docstrings, but to ensure they are properly
formatted if they're present.
For comments this adds checks that they don't contain code and verify
the formatting of comments with "TODO" tags.
As part of this, some commented out code which hasn't been touch in the
past 10 years gets removed as well.
The rules enabled enabled by this are:
- check formatting of existing docstrings (D200-)
- check comments for code (ERA)
- check formatting of TODO tags (TD001, TD004-)
This explicitly uses UTF-8 encoding when reading or writing files with
Python. This is necessary as the default locale varies between
operating systems.
This enables ruff rules which check for code which can be simplified to
improve readability.
The additionally rules getting enabled by this are:
- remove unnecessary nesting of if-statements (SIM102)
- use contextlib.suppress() for no-op exception handling (SIM105)
- use enumerate() for counting in loops (SIM113)
- use context managers for opening files (SIM115)
This enables some ruff rules to check for ambiguous and dead Python
code, which might cause unintended side-effects.
The enabled rules are:
- a bunch of rules related to shadowing of builtin structures (A)
- a bunch of rules checking for unused arguments (ARG)
- a rule checking for useless expressions (B018)
- a rule checking for unbound loop variables (B023)
- a rule checking redefined function parameters (PLR1704)
In the ruff config file added in #6954 explicitly selecting the ruff
rules to check was missed, resulting in ruff only checking a very small
subset of its available rules. That hasn't been desired, so this is the
first of a series of commits enabling more rules. In this PR all rules
whose violations can be either automatically fixed by ruff or are
trivial to fix manually get enabled. For the follow up PRs it's intended
to focus on one area of rules per PR to gradually improve the Python
code quality.
Stanislas Daniel Claude Dolcini