Chalet building libs proof of concept

[dimitre]

Hey @danoli3 I've uplaoded here a proof of concept of using Chalet to build libs.
it is already building and packaging and releasing assimp for macos
https://github.com/dimitre/ofChalet

[danoli3]

Cool proof of concept I just know we gonna hit some snags with that system

Definitely refer to the existing formulaes rather than make from scratch as the depths of issues resolved

"defines": [
				"BUILD_SHARED_LIBS=OFF",
				"ASSIMP_BUILD_TESTS=0",
				"ASSIMP_BUILD_SAMPLES=0",
				"ASSIMP_BUILD_3MF_IMPORTER=0",
				"CMAKE_COMPILE_WARNING_AS_ERROR=0",
				"CMAKE_OSX_DEPLOYMENT_TARGET=10.9",
				// "CMAKE_C_FLAGS=-arch arm64 -arch x86_64 -O3 -DNDEBUG -funroll-loops -Werror",
				"CMAKE_C_FLAGS=-fPIC",
				"CMAKE_CXX_FLAGS=-arch arm64 -arch x86_64 -O3 -DNDEBUG -funroll-loops -std=c++11 -Werror"
			],
			// "defaultRunArguments": [
			// 	"BUILD_SHARED_LIBS=OFF",
			// 	"ASSIMP_BUILD_TESTS=0",
			// 	"ASSIMP_BUILD_SAMPLES=0",
			// 	"ASSIMP_BUILD_3MF_IMPORTER=0"
			// ],
			"location": "${external:assimp}",
			"recheck": false,
			"rebuild": false

See

apothecary/apothecary/formulas/assimp.sh

Line 67 in 3d0997e

cmake .. ${DEFS} \

A few notes:

• Needs to be C17 / C++17 via Cmake: DCMAKE_C_STANDARD / DCMAKE_CXX_STANDARD
• Link to zlib rather than build subdependancy (and not use system libs as this will crash if arm64 and compiled on x86_64)
• Currently No toolchain use - ios/macos cmake toolchain is robust see if chalet can get that too
• Verify other missing flags
• No Pthread use (mac,unix)
• OSX Min: 10.15
• OSX Target: 14.0

See

if [[ "$TYPE" =~ ^(osx|ios|tvos|xros|catos|watchos)$ ]]; then
        echo "building $TYPE | $ARCH $PLATFORM"
        echo "--------------------" 
        ZLIB_ROOT="$LIBS_ROOT/zlib/"
        ZLIB_INCLUDE_DIR="$LIBS_ROOT/zlib/include"
        ZLIB_LIBRARY="$LIBS_ROOT/zlib/lib/$TYPE/$PLATFORM/zlib.a"   

        mkdir -p "build_${TYPE}_${PLATFORM}"
        cd "build_${TYPE}_${PLATFORM}"
        rm -f CMakeCache.txt || true
        find ./ -name "*.o" -type f -delete
        DEFS="
            -DASSIMP_BUILD_TESTS=0
            -DASSIMP_BUILD_SAMPLES=0
            -DASSIMP_BUILD_3MF_IMPORTER=0
            -DASSIMP_BUILD_ZLIB=OFF 
            -DASSIMP_WARNINGS_AS_ERRORS=OFF"

        cmake .. ${DEFS} \
            -DCMAKE_C_STANDARD=17 \
            -DCMAKE_CXX_STANDARD=17 \
            -DCMAKE_CXX_STANDARD_REQUIRED=ON \
            -DCMAKE_CXX_FLAGS="-DUSE_PTHREADS=1" \
            -DCMAKE_C_FLAGS="-DUSE_PTHREADS=1" \
            -DCMAKE_CXX_EXTENSIONS=OFF \
            -DBUILD_SHARED_LIBS=OFF \
            -DDEPLOYMENT_TARGET=${MIN_SDK_VER} \
            -DCMAKE_BUILD_TYPE=Release \
            -DCMAKE_INSTALL_PREFIX=Release \
            -DCMAKE_INCLUDE_OUTPUT_DIRECTORY=include \
            -DCMAKE_INSTALL_INCLUDEDIR=include \
            -DCMAKE_TOOLCHAIN_FILE=$APOTHECARY_DIR/toolchains/ios.toolchain.cmake \
            -DPLATFORM=$PLATFORM \
            -DENABLE_BITCODE=OFF \
            -DENABLE_ARC=OFF \
            -DCMAKE_POSITION_INDEPENDENT_CODE=TRUE \
            -DENABLE_VISIBILITY=OFF \
            -DCMAKE_PREFIX_PATH="${LIBS_ROOT}" \
            -DZLIB_ROOT=${ZLIB_ROOT} \
            -DZLIB_INCLUDE_DIR=${ZLIB_INCLUDE_DIR} \
            -DZLIB_LIBRARY=${ZLIB_LIBRARY} \
            -DCMAKE_VERBOSE_MAKEFILE=${VERBOSE_MAKEFILE} 

        cmake --build . --config Release

For many formulaes custom patches and cmakelists are packaged just make sure to look at what has been done else it will be a nightmare to debug. Can chalet embed files?

The one thing I like about chalet is the https://www.chalet-work.space/docs/adding-packages
cmake package generator but I tihnk this is as simple as packaging a cmake config file with binaries so makes it easier in cmakelists to find(PNG).

think your best bet is to first actually build apothecary from scratch with the build scripts and compare to the outputs

[dimitre]

I've worked a little bit more in ofChalet script to build dependencies and make a release in this repo
https://github.com/dimitre/ofChalet/blob/main/chalet.json

Still not the right file paths but already building some libraries for macOS
https://github.com/dimitre/ofChalet/releases/tag/v0.0.1

As a side project I'll try to build all OF dependencies in macOS using Chalet, let's see how it goes.
cc: @ofTheo

[roymacdonald]

this, looks very promising in order to get rid of apothecary.

[dimitre]

Great! it is missing just a few libs to be able to build everything in macOS.
PRs are welcome here if anybody wants to give a try advancing it

[roymacdonald]

I have never used chalet before, actually had no idea about it. Seems quite nice.
I was thinking how would it be if we would just use regular package management tools such as brew. I think you can have it to install specific versions. Put into a script and then move the files where they are expected to be.
Probably chalet can do something similar. What I like about the brew aproach is that you forget about all the requirements and what ever it is, that is already dealt with, while installing from git (as what I read chalet does) is not necesarily a straight forward way to do such.

[dimitre]

yes I've wondered about brew also, but it is nice OF uses static libs and applications are standalone as they are now.
Still wonder about building them from scratch so specific things like Neon / Simd optimizatinos can be enabled

[danoli3]

I just don’t trust Brew or any package manager generally as so easily hacked, as locked into sub dependancy he’ll where one of the subs is vunerable, I’ve hacked them; reported and reported fixed many times in the past. Depends what you making but if you can backdoor a Unity game cause they didn’t update freeimage you got urself a way to hack any device. Or libZ embedded with vulnerabilities is one of the easiest hack vector example. Apothecary modernization of last few years was to convert to cmake build scripts which included basically patching at source repos the multi platform side of the build to make it possible for multi platform ability. Since apothecary as is, builds for platforms and targets no other build system does, with the cmake lists. It would be good to also just get Linux working as it already can support all the GCC targets to the latest. Ai can help a lot I think just get the targets and cmake lists sorted. Obviously testing chalet and other more easily broken up build systems is always good as it can find issues with build scripts, consolidate builds for single targets but overall, I think apothecary as it is now, in bleeding, needs some bandages for the other platforms but is really not far away with the cmake system of just building for all targets. I had metal for example building on the plane before I got back from Sardinia. Android needs to be slightly updated too for new NDK. Anyways I’ll get my hat back on soon

…

On Sat, 12 Oct 2024 at 9:19 am, Dimitre ***@***.***> wrote: yes I've wondered about brew also, but it is nice OF uses static libs and applications are standalone as they are now. Still wonder about building them from scratch so specific things like Neon / Simd optimizatinos can be enabled — Reply to this email directly, view it on GitHub <#379 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGK2HDYPLIYKTYQHA3H4ILZ3BFF7AVCNFSM6AAAAABIKIENQGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMBYGE4DQOBXGQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[roymacdonald]

I have not dug into apothecary for quite a while but theo saying that it has given a good amount of headaches.
I see your point about package managers. @danoli3 since you've been working so much with Apothecary, can you make some sort of TO DO list, milestones, etc?

[dimitre]

@danoli3 I think all of CMAKE options can be passed to chalet also.
About patches, I personally think the best option (after trying to patch upstream) is keep a fork patched in some git url.
With this we can decrease complexity of building systems, share responsibility (mantaining a patched repo updated if needed for example)

but with a more flexble system I'm sure we are able to advance faster with more of us working on progress, so it is easier to adopt more modern libraries like OpenImageIO for ex.