文件列表:

CMakeLists.txt (5611, 2023-10-09)
LICENSE (1073, 2023-10-09)
_clang-format (4236, 2023-10-09)
build2 (0, 2023-10-09)
build2\kram.xcodeproj (0, 2023-10-09)
build2\kram.xcodeproj\project.pbxproj (182883, 2023-10-09)
build2\kram.xcworkspace (0, 2023-10-09)
build2\kram.xcworkspace\contents.xcworkspacedata (456, 2023-10-09)
build2\kramc.xcodeproj (0, 2023-10-09)
build2\kramc.xcodeproj\project.pbxproj (12349, 2023-10-09)
build2\kramv.xcodeproj (0, 2023-10-09)
build2\kramv.xcodeproj\project.pbxproj (42601, 2023-10-09)
gtlf (0, 2023-10-09)
gtlf\GLTF (0, 2023-10-09)
gtlf\GLTF\GLTF.h (1862, 2023-10-09)
gtlf\GLTF\GLTF.xcodeproj (0, 2023-10-09)
gtlf\GLTF\GLTF.xcodeproj\project.pbxproj (32774, 2023-10-09)
gtlf\GLTF\Headers (0, 2023-10-09)
gtlf\GLTF\Headers\Extensions (0, 2023-10-09)
gtlf\GLTF\Headers\Extensions\GLTFExtensionNames.h (1263, 2023-10-09)
gtlf\GLTF\Headers\Extensions\GLTFKHRLight.h (2148, 2023-10-09)
gtlf\GLTF\Headers\GLTFAccessor.h (1400, 2023-10-09)
gtlf\GLTF\Headers\GLTFAnimation.h (1763, 2023-10-09)
gtlf\GLTF\Headers\GLTFAsset.h (2741, 2023-10-09)
gtlf\GLTF\Headers\GLTFBinaryChunk.h (1259, 2023-10-09)
gtlf\GLTF\Headers\GLTFBuffer.h (1139, 2023-10-09)
gtlf\GLTF\Headers\GLTFBufferAllocator.h (1080, 2023-10-09)
gtlf\GLTF\Headers\GLTFBufferView.h (1219, 2023-10-09)
gtlf\GLTF\Headers\GLTFCamera.h (1632, 2023-10-09)
gtlf\GLTF\Headers\GLTFDefaultBufferAllocator.h (990, 2023-10-09)
gtlf\GLTF\Headers\GLTFEnums.h (6035, 2023-10-09)
gtlf\GLTF\Headers\GLTFImage.h (1514, 2023-10-09)
... ...

# kram, kram.exe C++11 main to libkram to create CLI tool. Encode/decode/info on PNG/KTX/KTX2/DDS files with LDR/HDR and BC/ASTC/ETC2. Runs on macOS/win. # libkram.a, libkram-ios.a, kram.lib C++11 library from 200 to 800KB in size depending on encoder options. Compiles for iOS (ARM), macOS (ARM/Intel), win (Intel). # hslparser Parses HLSL syntax and generates readable HLSL/MSL code without transpiling. DXC is then used to compile to spirv. https://github.com/alecazam/kram/tree/main/hlslparser # kram-thumb-win.dll Windows thumbnailer for DDS/KTX/KTX2. Go to build or bin folder. Install with "regsvr32.exe kram-thumb-win.dll". Uninstall with "regsvr32.exe /u kram-thumb-win.dll" https://github.com/alecazam/kram/tree/main/kram-thumb-win # kramv.app ObjC++ viewer for PNG/KTX/KTX2/DDS supported files from kram. Uses Metal compute and shaders, eyedropper, grids, debugging, preview. Supports HDR and all texture types. Mip, face, volume, and array access. No dmg yet, just drop onto /Applications folder. Runs on macOS (arm***/x***). Generates Finder thumbnails and QuickLook previews via modern macOS app extension mechanisms. Diagrams and screenshots can be located here: https://www.figma.com/file/bPmPSpBGTi2xTVnBDqVEq0/kram #### Releases includes builds for macOS (Xcode 14.3 - arm***/x***/clang) and Windows x*** (VS 2022 - x***/clang). kramv for macOS, kram for macOS/Win, libkram for macOS/iOS/Win, win-thumb-kram for Win. Android library via NDK is possible, but f16 support is spotty on devices. ### About kram kram is a wrapper to several popular encoders. Most encoders have sources, and have been optimized to use very little memory and generate high quality encodings at all settings. All kram encoders are currently CPU-based. Some of these encoders use SSE, and a SSE to Neon layer translates those. kram was built to be small and used as a library or app. It's also designed for mobile and desktop use. The final size with all encoders is under 1MB, and disabling each encoder chops off around 200KB down to a final 200KB app size via dead-code stripping. The code should compile with C++11 or higher. kram focuses on sending data efficiently and precisely to the encoders. kram handles srgb and premul at key points in mip generation. Source files use mmap to reduce memory, but fallback to file ops if that fails. Temp files are generated for output, and then renamed in case the app fails or is terminated. Mips are done in-place, and mip data is written out to a file to reduce memory usage. kram leaves out BC2 and etcrgb8a1 and PVRTC. Also BC6 still needs an encoder, and ASTC HDR encoding needs a bit more work to pull from half4/float4 source pixels. Many of the encoder sources can multithread a single image, but that is unused. kram is designed to batch process one texture per core/thread via a python script or a C++11 task system inside kram. This can use more ram depending on the core count. Texture-per-process and scripted modes currently both take the same amount of CPU time, but scripted mode is best if kram ever adds GPU-accelerated encoding. Similar to a makefile system, the script sample kramtexture.py uses modstamps to skip textures that have already been processed. If the source png/ktx/ktx2 is older than the output, then the file is skipped. Command line options are not yet compared, so if those change then use --force on the python script to rebuild all textures. Also a crc/hash could be used instead when modstamp isn't sufficient or the same data could come from different folders. ### About kramv kramv is a viewer for the BC/ASTC/ETC2 LDR/HDR KTX/KTX2/DDS textures generated by kram from LDR PNG and LDR/HDR KTX/KTX2/DDS sources. kramv decodes ASTC/ETC2 textures on macOS Intel, where the GPU doesn't support them. macOS with Apple Silicon supports all three formats, and doesn't need to decode. kramv uses ObjC++ with the intent to port to Windows C++ as time permits. Uses menus, buttons, and keyboard handling useful for texture triage and analysis. Drag and drop folders, bundles, and click-to-launch are supported. Recently used textures/folders/bundles are listed in the menu. The app currently shows a single document at a time. Subsequent opens reuse the same document Window. With bundles and folders, kramv will attempt to pair albedo and normal maps together by filename for the preview. Preview mode provides lighting, *** cutoff, and mip visuals for a given texture. Multiple shapes can help identify inconsistent normal maps. The u-axis advances counterclockwise, and v-axis advances down on the shapes. +Y OpenGL normals are assumed, not -Y DirectX convention. Lighting appears up and to the right when normal maps are correctly specified. In non-preview mode, point sampling in a pixel shader is used to show exact pixel values of a single mip, array, and face. Debug modes provide pixel analysis. KramLoader shows synchronous cpu upload to a private Metal texture, but does not yet supply the underlying KTXImage. Pinch-zoom and pan tries to keep the image from onscreen, and zoom is to the cursor so navigating feels intuitive. Compute shaders are used to sample a single pixel sample from the gpu texture for the eyedropper. This simplifies adding more viewable formats in the future, but there is not a cpu fallback. Normal.z is reconstructed and displayed in the hud, and linear and srgb channels are shown. ``` Formats - R/RG/RGBA 8/16F/32F, BC/ETC2/ASTC, RGB has limited import support Container Types - KTX, KTX2, PNG Content Types - Albedo, Normal, SDF, Height Debug modes - transparent, color, non-zero, gray, +x, +y, xy >= 1 Texture Types - 1darray (no mips), 2d, 2darray, 3d (no mips), cube, cube array decrement any advance/toggle listed below ? - show keyboard shortcuts P - toggle preview, disables debug mode, shows lit normals, and mips and filtering are enabled G - advance grid, none, pixel grid, block grid, atlas grid (32, ***, 128, 256), D - advance debug mode H - toggle hud U - toggle ui V - toggle vertical vs. horizontal buttons I - show texture info in overlay W - toggle wrap/address filter, scales uv from [0,1] to [0,2] and changes sampler to wrap/repeat A - show all - arrays, faces, slices and mips all on-screen 1/2/3/4 - show rgba channels in isolation, alpha as grayscale 7 - toggle signed/unsigned 8 - toggle shader premul, shader does this post-sample so only correct for point-sampling not preview R - reload from disk if changed, zoom to fit (at 1x with ) 0 - fit the current mip image to 1x, or fit view. (at 1x with ). Y - advance array F - advance face/slide M - advance mip S - advance shape mesh (plane, unit box, sphere, capsule), displays list, esc to get out of list C - advance shape channel (depth, uv, face normal, vtx normal, tangent, bitangent, mip) L - advance lighting mode (none, diffuse, diffuse + specular) T - toggle tangent generation ↓ - advance bundle/folder image (can traverse zip of ktx/ktx2 files), displays list, esc to get out of list → - advance counterpart (can see png, then encodes if viewing folders). Not yet finished. ``` ### Limitations Texture processing is complex and there be dragons. Just be aware of some of the limitations of kram and encoding. Lossy compression can only solve so much. ASTC and BC4-7 are newer formats, but all formats have time and quality tradeoffs. And encoder quality and issues remain. WebGL is still often stuck with older formats due to lack of implemented extensions. And all formats need endpoints/selectors reordering and zstd compression that KTX2 offers. I added a platform called "any" to with KTX2 holding UASTC+zstd and also ETC2/ASTC/BC+zstd. The scripts bundle up textures in an archive, but these should go to resource packs and asset catalogs which get signed and can have ODR applied. ``` GPU - none of the encoders use the GPU, so cpu threading and multi-process is used Rescale Filtering - 1x1 point filter Mip filtering - 2x2 box filter that's reasonable for pow2, and a non-linear filters for non-pow2 so there is no pixel shift done in linear space using half4 storage, in-place to save mem 1D array - no mip support due to hardware, no encoding 3D textures - no mip support, uses ASTC 2d slice encode used by Metal/Android, not exotic ASTC 3d format BC/ETC2/ASTC - supposedly WebGL requires pow2, and some implementation need top multiple of 4 for BC/ETC2 These formats are disabled: BC1 w/alpha - may re-enable 3 color for black + rgb BC2 - not useful ETC2_RGB8A1 - broken in ETC2 optimizations BC1 - artifacts from limits of format, artifacts from encoder, use BC7 w/2x memory ASTC LDR - rrr1, rrrg/gggr, rgb1, rgba must be followed to avoid endpoint storage, requires swizzles ASTC HDR - encoder uses 8-bit source image, need 16f/32f passed to encoder, no hw L+A mode R/RG/RGBA 8/16F/32F - use kram or ktx2ktx2+ktx2sc to generate supercompressed ktx2 R8/RG8/R16F - input/output rowBytes not aligned to 4 bytes to match KTX spec, code changes needed PVRTC - unsupported, no open-source encoders, requires pow2 size Containers PVR/Basis/Crunch - unsupoorted KTX - only uncompressed, mip levels are unaligned to block size from 4 byte length at chunk 0 metadata/props aren't standardized or prevalent libkram supports only text props for display in kramv KTX2 - works in kram and viewer, has aligned levels of mips when uncompressed, libkram supports None/Zlib/Zstd supercompression for read/write libkram does not support UASTC/BasisLZ yet DDS - works in kram and viewer, no mip compression, only BC and explicit formats, extended for ASTC/ETC kram/kramv only support newer DX10 style DDS format. Can view in Preview on macOS too. DDSHelper provides load/save. Pixel data ordered by chunk instead of by mips. No metadata. ``` ### An example pipeline ``` At build: * Lossless 8u/16F/32F KTX2 sources 2D, 2D, cube, and 2D atlas textures with zstd mips. Editing these in Photoshop/Gimp is still an issue. * Need to stop basing pipelines around PNG. This is a limited source format, but supported by editors. * Textures should be higher resolution, and checked into source control (git-lfs or p4). * Some sort of scripting to supply encoder preset index for textures. * Drop mips and encode to KTX using kram using the encoder preset * Build 2D array or 2D atalas assets and name/uv locations for those assets. * Convert KTX to KTX2 via ktx2ktx2 + ktx2sc as lossy encoded ETC2/BC/ASTC+zstd or UASTC+rdo+zstd * Bundle into an Asset Catalog (macOS/iOS), or resource pack (Android) for slicing and on-demand resource loading At runtime: * Mmap load all KTX2 textures read-only into memory using KramLoader. This is the backing-store. * Decode smaller faces/slices/array and their mips and upload to staging buffer and then gpu transfer/twiddle to private texture. * For example, can upload all lower mips in 1/3rd the space and skip all the top mips. Textures without mips cannot do this. * Use sparse texturing hardware or readback to indicate what mips are accessed by the hardware. * Purge top mips of large unused textures, but keep the bottom mips. * Upload top mips as needed and memory allows. ``` ### Building kram uses an explicit Xcode workspace and projects on Apple platforms. CMake can't clean, build workspaces, or handle app extensions needed for thumbnails/previews. I spent a lot of time trying to keep CMake working since it keeps kram from being tied to Xcode releases, but I also wanted to add better Finder integration. These all live in 'build2' to distinguish from the 'build' directory created for CMake. Like CMake, the cibuild.h script runs xcodebuild from the command line to generate all the libraries and apps into the bin directory. Note that Xcode has never been able to simultaneously open the same project included in different workspaces, so organize derivative workspaces carefully. ``` ./scripts/cibuild.h open build2/kram.xcworkspace ``` kram was using CMake to setup the projects and build, but it doesn't support workspaces, clean, or the thumbnail/preview extension linking. As a result, kramv.app, kram, and libkram are generated. So I'm building kramv.app and everything with a custom Xcode project now. The library can be useful in apps that want to include the decoder, or runtime compression of gpu-generated data. For Mac, the CMake build is out-of-source, and can be built from the command line, or debugged from the xcodeproj that is built. Ninja and Makefiles can also be generated from cmake, but remember to trash the CMakeCache.txt file. ``` mkdir build cmake .. -G Xcode cmake --build . --config Release or open kramWorkspace.xcodeproj or cmake --install ../bin --config Release ``` For Windows, CMake is still used. CMake build libkram, kramc, and kram-thumb-win.dll. This uses the clang compiler and x*** only. ``` mkdir build cmake .. -G "Visual Studio 16 2019" -T ClangCL -A x*** or cmake .. -G "Visual Studio 17 2022" -T ClangCL -A x*** cmake --build . --config Release or open kramWorkspace.sln or # not sure if install works on Win cmake --install ../bin --config Release ``` There are various CMake settings that control the various encoders. Each of these adds around 200KB. I tested with each of these turned off, so code should be isolated. The project will still show all sources. * -DATE=ON * -DASTCENC=ON * -DBCENC=ON * -DSQUISH=ON * -DETCTOOL=ON ### Commands * encode - encode/decode block formats, mipmaps, fast ***, premul, srgb, swizzles, LDR and HDR support, 16f/32f * decode - can convert any of the encode formats to s/rgba8 ktx files for display * info - dump dimensions and formats and metadata/props from png and ktx files * script - send a series of kram commands that are processed in a task system. Ammenable to gpu acceleration. ### Sample Scripts * kramTextures.py - python3 example that recursively walks directories and calls kram, or accumulates command and runs as a script * formatSources.sh - zsh script to run clang_format on the kram source directory (excludes open source) * fixfinder.sh - after updating /Applications, this flushes any cached copy of kram.app from LaunchServices To demonstrate how kram works, scripts/kramtextures.py applies platform-specific presets based on source filenames endings. The first form executes multiple kram processes with each file using a Python ThreadPoolExecutor. The second generates a script file, and then runs that in a C++ task system inside kram. The scripting system would allow gpu compute of commands, and more balanced memory and thread usage. ``` cd build # this will install "click" and other python package dependencies macOS pip3 install -r ../scripts/requirements.txt Win python3.exe -m pip install -U pip python3.exe -m pip install -r ../scripts/requirements.txt # Gen ktx using 8 processes, and bundles the results to a zip file ../scripts/kramTextures.py --jobs 8 -p android --bundle # this writes out a script of all commands and runs on threads in a single process ../scripts/kramTextures.py --jobs 8 -p ios --script ../scripts/kramTextures.py --jobs 8 -p mac --script --force ../scripts/kramTextures.py --jobs 8 -p win --script --force # Generate ktx2 output, and then bundle them into a zip ../scripts/kramTextures.py -p any -c ktx2 --bundle ../scripts/kramTextures.py -p android -c ktx2 --bundle --check # Generate dds output, and then bundle them into a zip ../scripts/kramTextures.py -p any -c dds --bundle ../scripts/kramTextures.py -p android -c dds --bundle --check # Generate textures for all platforms ../scripts/kramTests.sh ``` ### Sample Commands To test individual encoders, there are tests cases embedded into kram. Also individual textures can be processed, or the script records these commands and executes the encodes on multiple cores. ``` cd build ./Release/kram -testall ./Release/kram -test 1002 # for ktx ./Release/kram encode -f astc4x4 -srgb -premul -quality 49 -mipmax 1024 -type 2d -i ../tests/src/ColorMap-a.png -o ../tests/out/ios/ColorMap-a.ktx ./Release/kram encode -f etc2rg -signed -normal -quality 49 -mipmax 1024 -type 2d -i ../tests/src/collectorbarrel-n.png -o ../tests/out/ios/collectorbarrel-n.ktx ./Release/kram encode -f etc2r -signed -*** -quality 49 -mipmax 1024 -type 2d -i ../kram/tests/src/flipper-***.png -o ../tests/out/ios/flipper-***.ktx # for ktx (without and with zstd compression) ./Release/kram encode -f astc4x4 -srgb -premul -quality 49 -mipmax 1024 -type 2d -i ../tests/src/ColorMap-a.png -o ../tests/out/ios/ColorMap-a.ktx2 ./Release/kram encode -f astc4x4 -srgb -premul -quality 49 -mipmax 1024 -type 2d -zstd 0 -i ../tests/src/ColorMap-a.png -o ../tests/out/ios/ColorMap-a.ktx2 ``` ### Open Source Encoder Usage This app would not be possible without the open-source contributions from the following people and organizations. These people also inspired me to make this app open-source, and maybe this will encourage more great tools or game tech. kram includes the following encoders/decoders: | Encoder | Author | License | Encodes | Decodes | |----------|------------------|-------------|-----------------------------|---------| | BCEnc | Rich Geldreich | MIT | BC1,3,4,5,7 | same | | Squish | Simon Brown | MIT | BC1,3,4,5 | same | | ATE | Apple | no sources | BC1,4,5,7 ASTC4x4,8x8 LDR | LDR | | Astcenc | Arm | Apache 2.0 | ASTC4x4,5x5,6x6,8x8 LDR/HDR | same | | Etc2comp | Google | MIT | ETC2r11,rg11,rgb,rgba | same | | Explicit | Me | MIT | r/rg/rgba 8u/16f/32f | none | | Compress | AMD | MIT | BC6 | same | | GTLFKit | Warren Moore | MIT | none | gltf | ``` ATE Simple wrapper for encode/decode around this. Identifies encode/decode formats off version. BCEnc Commented out some unused code to suppress warnings Hooked in some SIMD code. Squish Simplified to single folder. Replaced sse vector with float4/a for ARM/Neon support. Astcenc v3.4 Provide rgba8u source pixels. Converted to 32f at tile level. Improved 1 and 2 channel format encoding (disable now). Avoid reading off end of arrays with padding. Support 2d array of src pixels instead of 3d. Force AVX and SSE path, and implement using sse2neon emlation on Neon. Etc2comp Simplified to single folder. Keep r11 and rg11 in integer space. 6x faster. Memory reduced signficantly. One block allocated. Single pass encoder works on one block at a time, and does not skip blocks. Multipass and multithread algorithm sorts vector, and split out blockPercentage from iteration count. RGB8A1 is broken. Optimized encodes by inlining CalcPixelError. 2x faster. Reduced memory by 4x and passing down rgba8u instead of rgba32f. Converted to 32f at tile level. ``` ### Open source usage kram includes additional open-source: | Library | Author | License | Purpose | |----------------|--------------------|-----------|---------------------------| | lodepng | Lode Vandevenne | MIT | png encode/decode | | SSE2Neon | John W. Ratcliff | MIT | sse to neon | | heman | Philip Rideout | MIT | parabola EDT for SDF | | TaskSystem | Sean Parent | MIT | C++11 work queue | | tmpfileplus | David Ireland | Moz 2.0 | fixes C tmpfile api ... ...

近期下载者：

相关文件：

评论：[我要评论] [举报此文件]

收藏者：