Unreal Engine C++: Skeletal Mesh doc sheet

UE SkeletalMesh documentation, reference sheet, cheat sheet, c++ - 01/2023 - #Unreal
Table of content


Official documentation

Skeletal Mesh editor

Skeletal Mesh Render Paths

Joint Sockets

Virtual Bones

Skin weights Profiles

Control rig



Disclaimer

The following may not be up to date, as the time of its writing, it should roughly apply to UE > 5.1. This document most certainly contains mistakes, use your own judgement and always test out things in small sample projects first. This sheet mainly caters towards programmers, it is essentially a shared memo, that I wrote from my Engineer's perspective, and presents how the code and data structures are layed out.

Intro

There are mainly 2 options to represent a mesh asset in Unreal, static mesh and skeletal mesh. While only static meshes (as of today) can use Nanite, skeletal meshes are the only way to represent skeletal deformation and to smoothly deform a triangular mesh. Note: you can associate static meshes to joints as a trick to use Nanite, this would only work for robot like rig where each part moves rigidly without deformation (this is exactly what Epic did in the first sample of UE 5.0 with the "ancient robot").

Before reading this sheet, I recommend to:

  • Get familiar with skeletal meshes in Unreal's UI first.
    (For instance importing some FBX file containing joint animations into Unreal's editor, opening Unreal's asset into the skeletal editor etc.)
  • Create a C++ project with an AActor equiped with a USkeletalMesh component to test out things with draw debug helper, ex: Code snippet to debug skeletal mesh transforms
    (if words such as "component", "AActor" sound like gibberish to you, you should definetly get familiar with those concepts first elsewhere)

Skeletal Mesh overview

Roughly speaking we have:

SkeletalMesh   
    LOD[0]    
    LOD[1]
    LOD[2]
        VertexBuffer[nb_vertices]
        // RefToLocal = WorldJointMat . WorldBindPoseMat^-1 = SkinningMatrixToApplyToVertices
        ReferenceToLocalMatrices[nb_joints_for_the_entire_skeleton] 

        MeshChunk[0] // MeshSection 0
            ChunkMatrices[nb_joints_influencing_the_section]
        MeshChunk[1] // MeshSection 1
            ChunkMatrices[nb_joints_influencing_the_section]

In other words, a mesh has several LOD levels (LOD0, LOD1, LOD2 etc.). Each LOD is composed of "mesh sections" also called "chunks". A section represents a sub-part of the whole mesh, and typically each section will be assigned to different materials. In the skeletal editor you can find a view of the LODs and theirs sections. You can choose to highlight or display specific sections:

vertices

Each mesh "section" (a.k.a "chunk") is rendered by a vertex factory. It also stores the offset in the vertex buffer where the chunk is located:

Which means a vertex may be described either by it's local index or global index: 

"Global index" <-> vertex index in the global vertex buffer (a concatenation of mesh sections)
"Local index"  <-> vertex index in a specific mesh section

Bones

A mesh section stores a buffer chunkMatrices[] which are the skinning matrices used to deform the section. So, each mesh section is rendered in a separate draw call, where only a sub-set of bones deforming this particular mesh section is considered and uploaded. Typically each section/chunk setup is done by its corresponding "vertex factory". There is also a global array of skinning matrices called ReferenceToLocalMatrices[] available at the LOD level. It's worth noting that the mapping between local (chunkMatrices[local index]) and global (ReferenceToLocalMatrices[global index]) joint indices is stored in: 


USkeletalMesh::
    FSkeletalMeshRenderData::
        FSkeletalMeshLODRenderData::
            FSkelMeshRenderSection::
                uint16 BoneMap[/*local bone index*/] // = global bone index.

Dealing with local and global indices of all sorts will be a recuring theme when tempering with skeletal meshes 😑.

Data Structure

Note 1: It is important to keep in mind Unreal runs multiple threads, 2 of which are the "GameThread" and the "RenderThread". To avoid race condition data used in the game thread is duplicated at each frame. So we have 2 data structure that keep in synch together.

Note 2: some of the data or methods are only available when running in the editor, the code wrapped into WITH_EDITOR or WITH_EDITORONLY_DATA macros is not available in the game runtime (after packaging/exporting the game). 

USkeletalMesh
    FSkeletalMeshModel      // Imported data (EDITOR ONLY)
    FSkeletalMeshRenderData // data prepared for rendering

Here is the more detailed view, both structure roughly follow the pattern ModelPointer->LODs[LODIndex].Sections[i]: 

// UE 4.26 – 5.1

USkeletalMesh : public USkinnedAsset // (SkeletalMesh.h)

    #if WITH_EDITOR
    // ---------------------------------------
    // FSkeletalMeshModel* GetImportedModel() override 

    FSkeletalMeshModel* ImportedModel; // (Rendering/SkeletalMeshModel.h) 
        FSkeletalMeshLODModel LODModels[]  // (Rendering/SkeletalMeshLODModel.h) 
            FImportedSkinWeightProfileData SkinWeightProfiles[]; 
    
            // (Rendering/SkeletalMeshLODModel.h)
            FSkelMeshSourceSectionUserData UserSectionsData[]                 
                ...

            // (Rendering/SkeletalMeshLODModel.h) 
            FSkelMeshSection Sections[] 
                //Index in the TMap: FSkeletalMeshLODModel::UserSectionsData
                int32 OriginalDataSectionIndex 
                // Skin weights, vertex position etc. 
                FSoftSkinVertex SoftVertices[]; 
                // Bones used by this section.                
                uint16 BoneMap[/*local index*/]; // = global idx
                ...
    #endif
    
    // ---------------------------------------------------
    // FSkeletalMeshRenderData* GetResourceForRendering() override
    // skinnedMeshComponent ->MeshObject->GetSkeletalMeshRenderData():
    // skeletalMeshComponent->MeshObject->GetSkeletalMeshRenderData():

    FSkeletalMeshRenderData* SkeletalMeshRenderData; // (SkeletalMeshRenderData.h) 
        FSkeletalMeshLODRenderData LODRenderData[];  // (SkeletalMeshLODRenderData.h)
            // FSkinWeightVertexBuffer* GetSkinWeightVertexBuffer()
            FSkinWeightVertexBuffer SkinWeightVertexBuffer;
            // Skin weight profile data structures, 
            // can contain multiple profiles and their runtime FSkinWeightVertexBuffer 
            FSkinWeightProfilesData SkinWeightProfilesData;
                       
            FSkelMeshRenderSection RenderSections[]; // (SkeletalMeshLODRenderData.h)
                // Bones used by this section. 
                // returns global bone index (ReferenceToLocalMatrices[]
                uint16 BoneMap[/*local index*/];

                uint32 NumVertices;

Components

USkeletalMesh is also available via various components: 

USkinnedMeshComponent : public UMeshComponent
    USkinnedAsset* GetSkinnedAsset() const;
    FSkeletalMeshObject* MeshObject; // #include "SkeletalRenderPublic.h"

USkeletalMeshComponent : public USkinnedMeshComponent
    USkeletalMesh* SkeletalMeshAsset;
    USkeletalMesh* GetSkeletalMeshAsset() const;


Look up vertices on CPU (FSkeletalMeshModel)

The code below shows how to extract vertex position for each mesh section (from SoftVertices). Can be executed on the game thread but this will only work for the editor code. Vertex positions represents the un-animated pose (a.k.a position in rest-pose / T-pose / reference pose): 

#include "Engine/SkeletalMesh.h"
#include "Rendering/SkeletalMeshModel.h" 

#if WITH_EDITOR
FTransform actorWorldTransform = this->GetActorTransform();
	
const FSkeletalMeshObject* skelMeshObject = MySkel->MeshObject;

USkeletalMesh* SkeletalMeshPtr = MySkel->GetSkeletalMeshAsset();
FSkeletalMeshModel* SourceModel = SkeletalMeshPtr->GetImportedModel();
// Get Active LOD
const int32 lodIndex = skelMeshObject->GetLOD();

// Or look up every LODs:
//for (int32 lodIndex = 0; lodIndex < SourceModel->LODModels.Num(); ++lodIndex)
{
    const FSkeletalMeshLODModel& lod = SourceModel->LODModels[lodIndex];
        
    uint32 numSections = lod.Sections.Num();
    for (uint32 sectionsIndex = 0; sectionsIndex < numSections; ++sectionsIndex)
    {
    	FSkelMeshSection& section = SourceModel->LODModels[lodIndex].Sections[sectionsIndex];
    
    	int32 nbVerts = section.NumVertices;
    	for (int32 vix = 0; vix < nbVerts; ++vix)
    	{
            const FVector3f& vertexPositionLocal = section.SoftVertices[vix].Position;	
    		
            const FVector& worldPosition =     
                actorWorldTransform.TransformPosition( FVector(vertexPositionLocal) );

            DrawDebugPoint(GetWorld(), worldPosition, 30, FColor(239, 220, 52), true);
    	}
    }
}
#endif


Look up vertices on CPU (FSkeletalMeshRenderData)


// Fill out your copyright notice in the Description page of Project Settings.

#pragma once

#include "CoreMinimal.h"
#include "GameFramework/Actor.h"
#include "MyActor.generated.h"

UCLASS()
class BONEMATRICESTEST_API AMyActor : public AActor
{
	GENERATED_BODY()
	
public:	
	// Sets default values for this actor's properties
	AMyActor();

	UPROPERTY(EditAnywhere);
	USkeletalMeshComponent* MySkel;

protected:
	// Called when the game starts or when spawned
	virtual void BeginPlay() override;

public:	
	// Called every frame
	virtual void Tick(float DeltaTime) override;

};

// Fill out your copyright notice in the Description page of Project Settings.


#include "MyActor.h"
#include "DrawDebugHelpers.h"
#include "SkeletalRenderPublic.h"

// Sets default values
AMyActor::AMyActor()
{
    // Set this actor to call Tick() every frame.  You can turn this off to improve performance if you don't need it.
    PrimaryActorTick.bCanEverTick = true;

}

// Called when the game starts or when spawned
void AMyActor::BeginPlay()
{
    Super::BeginPlay();

    if (MySkel == nullptr)
        return;
    
    FTransform actorWorldTransform = this->GetActorTransform();
    FTransform componentTransform = m_mySkel->GetComponentTransform(); 
    
    const FSkeletalMeshObject* skelMeshObject = MySkel->MeshObject;
    const FSkeletalMeshRenderData& renderData = skelMeshObject->GetSkeletalMeshRenderData();
    
    // Get Active LOD
    const int32 lodIndex = skelMeshObject->GetLOD();
    
    // Or look up every LODs:
    //for (int32 lodIndex = 0; lodIndex < renderData.LODRenderData.Num(); ++lodIndex)
    {
    	const FSkeletalMeshLODRenderData* lodRenderData = &renderData.LODRenderData[lodIndex];
    
    	uint32 numSections = lodRenderData->RenderSections.Num();
    	for (uint32 sectionsIndex = 0; sectionsIndex < numSections; ++sectionsIndex)
    	{
            const FSkelMeshRenderSection& section = lodRenderData->RenderSections[sectionsIndex];
            const FPositionVertexBuffer& vertexBuffer =
            lodRenderData->StaticVertexBuffers.PositionVertexBuffer;
            
            //TArray<FVector> positions;
            //positions.SetNum(section.NumVertices);
            for (uint32 vix = 0; vix < section.NumVertices; ++vix)
            {
                const int32 vertexBufferIndex = section.GetVertexBufferIndex() + vix;
                const FVector& vertexPositionLocal = 
                    (FVector)vertexBuffer.VertexPosition(vertexBufferIndex);
                
                const FVector& worldPosition = 
                    //actorWorldTransform.TransformPosition(vertexPositionLocal);
                    componentTransform.TransformPosition(vertexPositionLocal);

                //positions[vix] = vertexPosition;
                DrawDebugPoint(GetWorld(), worldPosition, 30, FColor(52, 220, 239), true);
            }
    	}
    }

}

// Called every frame
void AMyActor::Tick(float DeltaTime)
{
	Super::Tick(DeltaTime);
}

The code below shows how to extract vertex position for each mesh section. This can be executed on the game thread. The main ways to access FSkeletalMeshRenderData are:

 1) From a skinned mesh  (USkinnedMeshComponent->MeshObject;)
 2) From a skeletal mesh (USkeletalMeshComponent->MeshObject;)
 3) From USkeletalMesh   (USkeletalMesh->GetResourceForRendering();)



// 1) From skinned mesh
TObjectPtr<USkinnedMeshComponent> skinnedMesh = nullptr;
const FSkeletalMeshObject* skelMeshObject = skinnedMesh->MeshObject;
const FSkeletalMeshRenderData& renderData = skelMeshObject->GetSkeletalMeshRenderData();

// 2) From a skeletal mesh
TObjectPtr<USkeletalMeshComponent> skeletalMesh = nullptr;
const FSkeletalMeshObject* skelMeshObject = skeletalMesh->MeshObject;
const FSkeletalMeshRenderData& renderData = skelMeshObject->GetSkeletalMeshRenderData();

// 3) From USkeletalMesh
USkeletalMesh skeletalMesh = nullptr;
const FSkeletalMeshRenderData& renderData = skeletalMesh->GetResourceForRendering();

Local vertex index

Which allows you to look up vertices: 

const& FSkeletalMeshRenderData renderData = ...;

// Get Active LOD
const int32 lodIndex = skelMeshObject->GetLOD();

// Or look up every LODs:
for (int32 lodIndex = 0; lodIndex < renderData.LODRenderData.Num(); ++lodIndex)
{
    const FSkeletalMeshLODRenderData* lodRenderData = &renderData.LODRenderData[lodIndex];

    uint32 numSections = lodRenderData->RenderSections.Num();
    for (uint32 sectionsIndex = 0; sectionsIndex < numSections; ++sectionsIndex) 
    {
        const FSkelMeshRenderSection& section = lodRenderData->RenderSections[sectionsIndex];
        const FPositionVertexBuffer& vertexBuffer = 
            lodRenderData->StaticVertexBuffers.PositionVertexBuffer;
        
        TArray<FVector> positions;
        positions.SetNum(section.NumVertices);
        for (uint32 vix = 0; vix < section.NumVertices; ++vix)
        {
            const int32 vertexBufferIndex = section.GetVertexBufferIndex() + vix;
            const FVector& vertexPositionLocal = 
                (FVector)vertexBuffer.VertexPosition(vertexBufferIndex);

            // local position / object coordinates
            positions[vix] = vertexPositionLocal;

            // World position:
            FVector worldPosition = 
                skeletalMesh->GetActorTransform().TransformPosition(vertexPositionLocal);
        }
    }
}

This would be the un-animated position (a.k.a position in rest-pose / T-pose / reference pose) as skinning animation is usually computed on the GPU

Global vertex index

You can directly lookup the global buffer of vertices as well: 

FlushPersistentDebugLines(GetWorld());
     
FTransform actorWorldTransform = this->GetActorTransform();
FTransform componentTransform = m_mySkel->GetComponentTransform();

const& FSkeletalMeshRenderData renderData = ...;

// Get Active LOD
const int32 lodIndex = skelMeshObject->GetLOD();

// Or look up every LODs:
//for (int32 lodIndex = 0; lodIndex < renderData.LODRenderData.Num(); ++lodIndex)
{
    
    const FSkeletalMeshLODRenderData* lodRenderData = &(renderData->LODRenderData[lodIndex]);

    const FPositionVertexBuffer& vertexBuffer =
        lodRenderData->StaticVertexBuffers.PositionVertexBuffer;
    
    int32 nbVerts = lodRenderData->GetNumVertices();
    for (int32 vix = 0; vix < nbVerts; ++vix)
    {
        const FVector& vertexPositionLocal =
            (FVector)vertexBuffer.VertexPosition(vix);

        FVector worldPosition =
            m_mySkel->GetComponentTransform().TransformPosition(vertexPositionLocal);
            //this->GetActorTransform().TransformPosition(vertexPositionLocal);

        DrawDebugPoint(GetWorld(),
            worldPosition,
            10,
            FColor(239, 220, 52),
            true);
    }
}

Look up LOD & Sections (FSkeletalMeshModel)

This would be only available in the Editor and not the game runtime after packaging: 


#include "Engine/SkeletalMesh.h" 
#include "Rendering/SkeletalMeshModel.h" 

TObjectPtr<USkeletalMeshComponent> SkeletalMesh;
USkeletalMesh* SkeletalMeshPtr = SkeletalMesh->GetSkeletalMeshAsset(); 
FSkeletalMeshModel* SourceModel = SkeletalMeshPtr->GetImportedModel();
for (int32 LODIndex = 0; LODIndex < SourceModel->LODModels.Num(); ++LODIndex) {
for (int32 i = 0; i < SourceModel->LODModels[LODIndex].Sections.Num(); ++i) {
        SourceModel->LODModels[LODIndex].Sections[i].MaterialIndex = NewIndex;
        ++NewIndex;
    }
}


FBX to UE data mapping

Keywords to look up to find things related to skin cluster import: 

FbxCluster* Cluster = Skin->GetCluster(ClusterIndex);
lClusterCount =( (FbxSkin *)FbxMesh->GetDeformer(i, FbxDeformer::eSkin))->GetClusterCount();

Mesh sections

Consider a FBX file generated from Maya with several mesh objects and materials. It seems UE will regroup any geometry with the same material under a mesh section.

Duplicated vertices

When importing a mesh, for instance via .fbx, vertices may get duplicated (a.k.a split). For instance, to allow the definition of several normals at the same vertex, a typical way to light sharp edges. It can also occur when we need to associate multiple uv coordinates for the same vertex, this happens whenever a vertex lies on a texture seam. In other words, for each mesh object (a.k.a mesh section in UE), the vertex count displayed in Maya (before exporting) is likely to be different to the number of vertices of each mesh section you will find in UE's USkeletalMesh.

This means at least two things:

  • Importing vertex data likely requires you to mirror how UE duplicates vertex data.
      • If you import new vertex data into UE, say exporting a vertex weight map (e.g. a float that represent the mass of each vertex) from a DCC tool like Maya and importing it into UE, your custom C++ UE importer will likely need to duplicate vertex data exactly like UE does in his (ex FBX) import code.
  • In vertex-based simulation it's likely you need to only treat original vertices
      • For instance if you implement a custom cloth simulation, you would want to base the simulation on the original topology to avoid breaking the simulation and optimize computation. Only after doing the accurate and heavy computation would you copy the final vertex position to the duplicated vertices for rendering.

Fortunatly, UE keeps the mapping between imported vertices and render vertices (i.e. duplicated vertices).

Terminology

It's quite a confusing topic, so we will define our own terminology (different from official UE notations). Let's consider 2 vertex buffers, we name the associated vertex index as follows: 

TArray<FVector> VertexListWhenImported     <- [importVertIdx]; // Same as DCC tool, maya etc.
TArray<FVector> VertexListAfterDuplication <- [renderVertIdx]; // Duplicated and ready for rendering.

In addition, we can convert both index type to either: 

"Global index" <-> vertex index inside the LOD buffer of concatenated mesh sections
"Local index"  <-> vertex index inside a specific mesh section

OverlappingVertices & DuplicatedVerticesBuffer buffers

Warning: In this paragraph everything is expressed in local renderVertIdx (as opposed to importedVertIdx).

Let's describe the buffers: 
TMap<int32, TArray<int32>> FSkelMeshSection::OverlappingVertices;
// Same as above but with a "flat" memory layout
FDuplicatedVerticesBuffer  FSkelMeshRenderSection::DuplicatedVerticesBuffer

Here is where both buffers are accessible relative to the skeletal mesh: 

// UE 4.26 – 5.1
USkeletalMesh
    // ------------------
    // GetImportedModel() 
    // (Editor only)
    FSkeletalMeshModel* ImportedModel; 
        FSkeletalMeshLODModel LODModels[]  
            FSkelMeshSection Sections[]
                TMap<int32, TArray<int32>> OverlappingVertices;
     
    // -------------------------  
    // GetResourceForRendering()
    // Both editor & game runtime
    FSkeletalMeshRenderData* SkeletalMeshRenderData; 
        FSkeletalMeshLODRenderData LODRenderData[]; 
            FSkelMeshRenderSection RenderSections[];
                FDuplicatedVerticesBuffer DuplicatedVerticesBuffer; // GPU friendly memory layout

They are both the same with different memory layout. The map OverlappingVertices associates a vertex index (in local renderVertIdx) to a sub-list of vertex indices (in local renderVertIdx) with same coordinates.

OverlappingVertices

In other words: 

TMap<int32, TArray<int32>> FSkelMeshSection::OverlappingVertices;

Maps any vertex index to all vertices that share the same position (all indices are local renderVertIdx). 

OverlappingVertices[localRenderVertIdx_i] 
                 = 
sub list of localRenderVertIdx_j that share the same coordinates as localRenderVertIdx_i;

FDuplicatedVerticesBuffer

The memory flat version of OverlappingVertices is FDuplicatedVerticesBuffer FSkelMeshRenderSection::DuplicatedVerticesBuffer; 

FIndexLengthPair
    int Index;
    int Length;

FDuplicatedVerticesBuffer    
    // Look up table:

    // Maps each vertex to a sub array stored in ‘FDuplicatedVerticesBuffer::DupVertData[]’
    // DupVertIndexData[localRenderVert] = 
    //     {.Length == number of duplicated vertices; .Index == start index in 'DupVertData[]'}
    TSkeletalMeshVertexData<FIndexLengthPair> DupVertIndexData;
    

    // flat data:

    // Concatanation of all 'TArray<int32>' from 'TMap<int32, TArray<int32>> OverlappingVertices'

    // Example:
    // let's rename: OverlappingVertices := Ov;
    // DupVertData[] = 
    //    {Ov[ 0 ][0], Ov[ 0 ][1], Ov[ 0 ][2]; 
    //     Ov[ 1 ][0], Ov[ 1 ][1], Ov[ 1 ][2], Ov[1][3], Ov[1][4];
    //     ...
    //     Ov[vix][0], Ov[vix][1], Ov[vix][2];
    //     ... 
    //     }
    TSkeletalMeshVertexData<uint32> DupVertData;

Sample code to look up FDuplicatedVerticesBuffer:

// All vertex indices here refer to render vertices 
// (as opposed to imported vertices)
bool notDup = false;
// Look up every vertices of the current mesh section:
for (uint32 vix = 0; vix < RenderSection.NumVertices; ++vix)
{    
    const int32 globalVertIdx = RenderSection.GetVertexBufferIndex() + vix;
    const auto& listDescriptor = RenderSection.DuplicatedVerticesBuffer.DupVertIndexData[vix];
    const FPositionVertexBuffer& vertexBuff = LodRenderData->StaticVertexBuffers.PositionVertexBuffer
    const FVector& VertexPosition = (FVector)vertexBuff.VertexPosition(globalVertIdx);			
    // look up sub list of duplicated vertices:
    for (uint32 l = listDescriptor.Index; l < listDescriptor.Length; ++l) 
    {
        const int32 localDupIdx = RenderSection.DuplicatedVerticesBuffer.DupVertData[l];
        int32 globalRenderVertIdx = RenderSection.GetVertexBufferIndex() + localDupIdx;
        const FVector& DupPosition = (FVector)vertexBuff.VertexPosition(globalRenderVertIdx);
        FVector res = DupPosition - VertexPosition;
        if (res.Length() > 0.000001f)
            notDup = true;
    }			
}
// should turn out true if the mesh has duplicated vertices:
notDup;

Bonus: FDuplicatedVerticesBuffer is computed with: 

void FDuplicatedVerticesBuffer::Init(
        const int32 NumVertices, 
        const TMap<int32, 
        TArray<int32>>& OverlappingVertices);

NewRenderSection.DuplicatedVerticesBuffer.Init(
        ModelSection.NumVertices, 
        ModelSection.OverlappingVertices);

MeshToImportVertexMap

Map between importVertIdx and renderVertIdx is found in: 

// Warning: global index. 
// (as opposed to local indices relative to a mesh section)
FSkeletalMeshLODModel::MeshToImportVertexMap[renderVertIdx] = importVertIdx

Here is where both buffers are accessible relative to the skeletal mesh: 

// UE 4.26 – 5.1
USkeletalMesh
    // ------------------
    // GetImportedModel() 
    // (Editor only)
    FSkeletalMeshModel* ImportedModel; 
        FSkeletalMeshLODModel LODModels[]  
            MeshToImportVertexMap[renderVertIdx]            

            FSkelMeshSection Sections[]
                ...
     
    // -------------------------  
    // GetResourceForRendering()
    // Both editor & game runtime
    FSkeletalMeshRenderData* SkeletalMeshRenderData; 
        ...

That you typically access as follows: 

TObjectPtr<USkeletalMeshComponent> SkeletalMesh
const USkeletalMesh* Mesh = SkeletalMesh->GetSkeletalMeshAsset();
const FSkeletalMeshModel* SkeletalMeshModel = Mesh->GetImportedModel();
int32 importVertIdx = SkeletalMeshModel->LODModels[LODIndex].MeshToImportVertexMap[renderVertIdx];

Code sample: 

const FSkeletalMeshModel* SkeletalMeshModel;
// Look up LODs:
for (int32 LodIndex = 0; LodIndex < LODSkinWeights.Num(); ++LodIndex)
{    
    const FSkeletalMeshLODModel& LODModel = SkeletalMeshModel->LODModels[LodIndex];
    //Note:
    uint32 NumFbxVerts = LODModel.MaxImportVertex;

    // Lookup global renderVertIdx
    for (uint32 renderVertIdx = 0; renderVertIdx < LODModel.NumVertices; ++renderVertIdx) 
    {						
        // Warning: both importVertIdx and renderVertIdx are global here:
        const int32 importVertIdx = LODModel.MeshToImportVertexMap[renderVertIdx];
        ...
    }
}

Bonus:

When importing a FBX file: 

        FMeshUtilities::BuildSkeletalModelFromChunks(){
            LODModel.MeshToImportVertexMap.Add(RawVertIndex);
        }
        
        // Which is based on the data LODPointToRawMap / PointToRawMap :
        
        TArray<int32> LODPointToRawMap;
        SkeletalMeshImportData.CopyLODImportData(
                LODPoints, 
                LODWedges, 
                LODFaces, 
                LODInfluences, 
                LODPointToRawMap)
        {
            LODPointToRawMap = PointToRawMap;
        }
        
        // PointToRawMap is filled when duplicating vertices of a smoothing group:
        void FSkeletalMeshImportData::SplitVerticesBySmoothingGroups(){
            PointToRawMap[NewPointIndex] = p;
        }
        
        int32 UnFbx::FFbxImporter::DoUnSmoothVerts(
                FSkeletalMeshImportData &ImportData, 
                bool bDuplicateUnSmoothWedges)
        {
            ImportData.PointToRawMap[NewPointIndex] = p;
        }

This may happen in other places (My intuition tells me it may happen for UV seams) Note that as a vertex gets split/duplicated other data such as skin weights are also duplicated.

Convert "renderVertIdx" from "global" to "local"

To convert a renderVertIdx from global to a local index relative to a mesh section use: 

void FSkeletalMeshLODModel::GetSectionFromVertexIndex(
        int32 InVertIndex,      // global renderVertIdx
        int32& OutSectionIndex, // Section's index
        int32& OutVertIndex)    // *local* renderVertIdx (relative to OutSectionIndex)

Code sample to Look up LODs and global renderVertIdx inside to then convert to local renderVertIdx: 

const FSkeletalMeshModel* SkeletalMeshModel;
// Look up LODs:
for (int32 LodIndex = 0; LodIndex < LODSkinWeights.Num(); ++LodIndex) {
    const FSkeletalMeshLODModel& LODModel = SkeletalMeshModel->LODModels[LodIndex];
    // Lookup global renderVertIdx
    for (uint32 renderVertIdx = 0; renderVertIdx < LODModel.NumVertices; ++renderVertIdx) {
        // Convert to local indices relative to the mesh section/chunk:
        int32 sectionIdx;
        int32 localrenderVertIdx;
        LODModel.GetSectionFromVertexIndex(
                    /*in:  */ renderVertIdx,
                    /*out: */ sectionIdx,
                    /*out: */ localrenderVertIdx);
        ...
    }
}

Convert "importVertIdx" from "global" to "local"

Function to operate the conversion global importVertIx to local importVertIx: 


TTuple<int32/*out SectionIndex*/, int32/*out local ImportVertIdx*/> 
ToLocalFBXIndex(
    int32 globalImportVertIdx,
    const FSkeletalMeshLODModel& LODModel)
{
    int32 outSectionIndex = -1;
    int32 outLocalImportVertIndex = -1;
    
    check( LODModel.ImportedMeshInfos.Num() == LODModel.Sections.Num() );
    int32 numChunks = LODModel.ImportedMeshInfos.Num();
    int32 vertCount = 0;	
    for (int32 sectionCount = 0; sectionCount < numChunks; sectionCount++)
    {		
        const FSkelMeshImportedMeshInfo& section = LODModel.ImportedMeshInfos[sectionCount];
        outSectionIndex = sectionCount;
        check(section.StartImportedVertex == vertCount);
        // Is it in section's range?
        if (globalImportVertIdx < vertCount + section.NumVertices)
        {
            outLocalImportVertIndex = globalImportVertIdx - vertCount;
            return MakeTuple(outSectionIndex, outLocalImportVertIndex);
        }
        vertCount += section.NumVertices;
    }
    
    return MakeTuple(outSectionIndex, outLocalImportVertIndex);
}

{
    int32 localFBXVertIdx;
    int32 sectionFBXIdx;
    Tie(sectionFBXIdx, localFBXVertIdx) = ToLocalFBXIndex(globalFBXIdx, LODModel);
}


Mesh sections by name

You can find the names of the sub-meshes (a.k.a section or chunk) that compose a USkeletalMesh. For instance, when exporting several objects from Maya into a .FBX file each object's name is associated to a UE mesh section as follows: 

const FSkeletalMeshLODModel& LODModel
int32 NumChunks = LODModel.ImportedMeshInfos.Num();
for (int32 SectionCount = 0; SectionCount < NumChunks; SectionCount++)
{		
    const FSkelMeshImportedMeshInfo& section = LODModel.ImportedMeshInfos[SectionCount];
    FName = section.Name;
}


Joint data

Some useful methods: 

// USkinnedMeshComponent / SkinnedMeshComponent.h

// Total number of joints
int32 USkinnedMeshComponent::GetNumBones() const
FName USkinnedMeshComponent::GetBoneName(int32 BoneIndex) const
int32 USkinnedMeshComponent::GetBoneIndex( FName BoneName) const
FName USkinnedMeshComponent::GetParentBone( FName BoneName ) const

List names

List every joint names and associated indices: 

USkeletalMeshComponent* skeletalMesh = ...;
TArray<FName> boneNames;
skeletalMesh->GetBoneNames(boneNames);

// Display bone names and indices
for (int32 boneIndex = 0; boneIndex < boneNames.Num(); ++boneIndex)
{
    const FName& boneName = boneNames[boneIndex];
    UE_LOG(LogTemp, Warning, 
        TEXT("Array Index: %d, Bone Index: %d, Bone Name: %s"), 
        boneIndex, 
        skeletalMesh->GetBoneIndex(boneName), 
        *boneName.ToString()
    );
}

TODO: check wether this list includes the "virtual bones" and "joint sockets" or not.


Matrices, transformations etc.

Remember that FMatrix multiplication is applied from left to right in Unreal's cpu code:
vec x M1 x M2 x ... where M1 is applied first then M2 etc.


Get bone local transformation (according to its parent bone): 

// GetBoneSpaceTransforms()
TArray<FTransform> localBoneTransforms = skeletalMesh->GetBoneSpaceTransforms();
FTransform localTransform = localBoneTransforms[boneIndex];
FMatrix44f localMatrix    = FMatrix44f( localTransform.ToMatrixWithScale() );

Get global transformations of the bone (i.e. world coordinates of the scene)

// GetBoneMatrix(int idx) 
// GetBoneTransform(int idx)
// Both lines are equivalent:
FMatrix44f mat = FMatrix44f( skeletalMesh->GetBoneMatrix(boneIndex) );
FMatrix44f mat = FMatrix44f( skeletalMesh->GetBoneTransform(boneIndex).ToMatrixWithScale() );

Get transformation of a bone in component space (object coordinates)

// GetBoneTransform(int idx, FTransform::Identity)
// Transformation of the joint expressed relative to the skeletal mesh object coordinates, 
// as opposed to the scene (world) coordinates
FTransform tr = skeletalMesh->GetBoneTransform(boneIndex, FTransform::Identity);
FMatrix44f mat = FMatrix44f( tr.ToMatrixWithScale()  );

/*
    It's recommended to use the above method as they also handle corner cases,
    but keep in mind finding a joint's matrix in component space is equivalent to:
    (yes this naming is confusing since it does not mention "joint" or "bone"...)
*/
skeletalMesh->GetComponentSpaceTransforms()[BoneIdx].ToMatrixWithScale();

// Global/world coordinates would be then:
skeletalMesh->GetComponentSpaceTransforms()[BoneIdx].ToMatrixWithScale() * skeletalMesh->GetComponentTransform().ToMatrixWithScale();

Joint sockets

The joints of a skeletal mesh can be extended with "sockets". A socket is a special type of joint that you insert in the skeletal hierarchy and is parented to some original joint of your imported model. This is useful to bind props for instance. See official doc about the UI to use sockets.

You can access the transformation value of a socket: 

/*
    Uncomplete list of API to interact with sockets:
*/

// Some accessors available from a component:
RootComponent->USceneComponent::GetSocketLocation()


USkeletalMesh {

    TArray<USkeletalMeshSocket*>& GetMeshOnlySocketList();

    // all sockets from this mesh plus all non-duplicates from the skeleton
    TArray<USkeletalMeshSocket*> USkeletalMesh::GetActiveSocketList() const

    // Add a skeletal socket object to this SkeletalMesh, 
    // and optionally promotes it to USkeleton socket.
    void AddSocket(USkeletalMeshSocket* inSocket, bool bAddToSkeleton=false);

    // If there are multiple sockets with the same name, will return the first one.
    USkeletalMeshSocket* FindSocket(FName inSocketName) const override;    
    
    USkeletalMeshSocket* FindSocketAndIndex(FName inSocketName, int32& outIndex) const;
    
    USkeletalMeshSocket* FindSocketInfo(
                                FName inSocketName, 
                                FTransform& outTransform, 
                                int32& outBoneIndex, 
                                int32& outIndex) const;
  
    int32 NumSockets() const;
    
    USkeletalMeshSocket* GetSocketByIndex(int32 index) const;
    
    // Called to rebuild an out-of-date or invalid socket map
    void RebuildSocketMap();
}

USkeletalMeshSocket {


}

Virtual bones

Inside the Skeleton Editor, you can add Virtual Bones to the existing skeleton. This allows adding joints for aiming or IK (inverse kinematics) directly from inside the Editor, as opposed to go back to your DCC software (e.g. Maya), edit and then reimport all the animations to fix the animation data with the new joints included.

Code snippet to debug skeletal mesh transforms

UCLASS()
class BONEMATRICESTEST_API AMyActor : public AActor
{
	GENERATED_BODY()
public:	
	// Sets default values for this actor's properties
	AMyActor();

	UPROPERTY(EditAnywhere);
	USkeletalMeshComponent* MySkel;

	UPROPERTY(EditAnywhere, Category = "Locations")
	FVector LocationOne;
protected:
	// Called when the game starts or when spawned
	virtual void BeginPlay() override;
public:	
	// Called every frame
	virtual void Tick(float DeltaTime) override;
};

#include "MyActor.h"
#include "DrawDebugHelpers.h"

// Sets default values
AMyActor::AMyActor()
{
    PrimaryActorTick.bCanEverTick = true;
    LocationOne = FVector(0, 0, 0);
}

// Called when the game starts or when spawned
void AMyActor::BeginPlay()
{
    Super::BeginPlay();
    //MySkel->GetBoneMatrix();
    int32 idx = MySkel->GetBoneIndex("hand_l");
    if (idx > -1)
    {
        FTransform identity = FTransform::Identity;
        FTransform tr = MySkel->GetBoneTransform(idx, FTransform::Identity);
        FMatrix m;
	
        FVector Loc = tr.GetLocation();
        FQuat Rot = tr.GetRotation();
        FRotator EulerRot = Rot.Rotator();
        FVector Scale = tr.GetScale3D();
        LocationOne = Loc; 

        DrawDebugPoint(GetWorld(), LocationOne, 30, FColor(52, 220, 239), true);
        DrawDebugCoordinateSystem(GetWorld(), Loc, EulerRot, 30.f, true);	
    }
}

// Called every frame
void AMyActor::Tick(float DeltaTime)
{
    Super::Tick(DeltaTime);
}


Skinning

See skinning cheat sheet for general math.

Bind pose / T-Pose

In Unreal Engine the reference pose of a skeletal mesh designates:

  • "bind pose"
  • "rest pose"
  • "T-Pose"
(all terms are equivalent)

Sometimes abreviated as ref you may see names like refskeleton, refmatrix or ReferenceToLocal as a way to signal it makes use of the reference pose.

C++ to find the reference pose (a.k.a bind pose)

#include "ReferenceSkeleton.h"
FReferenceSkeleton refSkeleton = USkeletalMesh->RefSkeleton;

FReferenceSkeleton stores the T-pose as a list of transforms in local coordinates (relative to parent joint): 

#include "ReferenceSkeleton.h"

FTransform getRefPoseBoneTransforms(USkeletalMeshComponent* skelMesh, FName boneName) 
{
    FTransform boneTransform;
    const FReferenceSkeleton& RefSkel = SkelMesh->GetSkinnedAsset()->GetRefSkeleton();
    boneTransform = refSkel.GetRefBonePose()[refSkel.FindBoneIndex(boneName)];
    return boneTransform;
}

// Almost equivalent to the above there is a built in method to retreive the bind pose:
FMatrix USkeletalMesh::GetRefPoseMatrix( int32 BoneIndex ) const;
{
    FTransform BoneTransform = GetRefSkeleton().GetRawRefBonePose()[BoneIndex];
    // Make sure quaternion is normalized!
    BoneTransform.NormalizeRotation();
    return BoneTransform.ToMatrixWithScale();
}
// The issue is that it relies on GetRawRefBonePose(),
// The doc says: 'RawRefBone' is the data as imported. 
// 'RefBone' also contains "virtual bones"
// so this version may not handle some corner case.

/*
Virtual bone documentation:
https://docs.unrealengine.com/4.27/en-US/AnimatingObjects/SkeletalMeshAnimation/Persona/VirtualBones/
*/

Now you can compute from this the bind-pose in component space (object coordinates). Which means expressing each joint's transform relative to the component / object coordinates: 

/// @return The bone transform for bone 'boneIdx' at rest-pose in component space
FTransform get_ref_pose_single_bone_comp_space(const FReferenceSkeleton& inSkel, int32 boneIdx)
{
    // Local transform (relative to parent joint)
    FTransform resultBoneTransform = inSkel.GetRefBonePose()[boneIdx];
    auto refBoneInfo = inSkel.GetRefBoneInfo();
    while (boneIdx)
    {
        resultBoneTransform *= inSkel.GetRefBonePose()[refBoneInfo[boneIdx].ParentIndex];
        boneIdx = refBoneInfo[boneIdx].ParentIndex;
    }
    
    return resultBoneTransform;
}


/// @return the list of bone transforms at rest-pose in component space
TArray<FTransform> get_ref_pose_bone_comp_space(const USkeletalMeshComponent* inSkelComp)
{
    const FReferenceSkeleton& refSkeleton = inSkelComp->GetSkinnedAsset()->GetRefSkeleton();
    const int32 poseNum = refSkeleton.GetRefBonePose().Num();
    TArray<FTransform> outResult;
    outResult.Reset();
    outResult.AddUninitialized(poseNum);
    // Compute global transform for each joint?
    for (int32 i = 0; i < poseNum; i++) {
        outResult[i] = get_ref_pose_single_bone_comp_space(refSkeleton, i);
    }
    
    return outResult;
}

To get the world coordinates of the bones, for instance in an AActor, you would need to multiply by the transform expressing the world coordinates of that actor: 

const USkeletalMeshComponent* inSkelComp = ...;
const FReferenceSkeleton& refSkel = inSkelComp->GetSkinnedAsset()->GetRefSkeleton();

FTransform actorWorldTransform = this->GetActorTransform();
FTransform tr = get_ref_pose_single_bone_comp_space(
                    refSkel, 
                    refSkel.FindBoneIndex(name));

// World coordinates:
tr = tr * actorWorldTransform;

original article. 

// Fill out your copyright notice in the Description page of Project Settings.

#pragma once

#include "CoreMinimal.h"
#include "GameFramework/Actor.h"
#include "MyActor.generated.h"

UCLASS()
class BONEMATRICESTEST_API AMyActor : public AActor
{
	GENERATED_BODY()
	
public:	
	// Sets default values for this actor's properties
	AMyActor();

	UPROPERTY(EditAnywhere);
	USkeletalMeshComponent* MySkel;	

protected:
	// Called when the game starts or when spawned
	virtual void BeginPlay() override;

public:	
	// Called every frame
	virtual void Tick(float DeltaTime) override;

};


// Fill out your copyright notice in the Description page of Project Settings.


#include "MyActor.h"
#include "DrawDebugHelpers.h"
#include "SkeletalRenderPublic.h"
#include "ReferenceSkeleton.h"

// Sets default values
AMyActor::AMyActor()
{
 	// Set this actor to call Tick() every frame.  You can turn this off to improve performance if you don't need it.
	PrimaryActorTick.bCanEverTick = true;		
	
}


FTransform GetRefPoseBoneTransforms(USkeletalMeshComponent* SkelMesh, FName BoneName) {
	FTransform BoneTransform;
	const FReferenceSkeleton& RefSkel = SkelMesh->GetSkinnedAsset()->GetRefSkeleton();
	BoneTransform = RefSkel.GetRefBonePose()[RefSkel.FindBoneIndex(BoneName)];
	return BoneTransform;
}


/// @return The bone transform for bone 'boneIdx' at rest-pose in component space
FTransform get_ref_pose_single_bone_comp_space_transform(const FReferenceSkeleton& inSkel, int32 boneIdx)
{
	// Local transform (relative to parent joint)
	FTransform resultBoneTransform = inSkel.GetRefBonePose()[boneIdx];
	auto refBoneInfo = inSkel.GetRefBoneInfo();
	while (boneIdx)
	{
		resultBoneTransform *= inSkel.GetRefBonePose()[refBoneInfo[boneIdx].ParentIndex];
		boneIdx = refBoneInfo[boneIdx].ParentIndex;
	}

	return resultBoneTransform;
}


/// @return the list of bone transforms at rest-pose in component space
TArray<FTransform> get_ref_pose_bone_comp_space_transform(const USkeletalMeshComponent* inSkelComp)
{
	const FReferenceSkeleton& refSkeleton = inSkelComp->GetSkinnedAsset()->GetRefSkeleton();
	const int32 poseNum = refSkeleton.GetRefBonePose().Num();
	TArray<FTransform> outResult;
	outResult.Reset();
	outResult.AddUninitialized(poseNum);
	// Compute global transform for each joint?
	for (int32 i = 0; i < poseNum; i++) {
		outResult[i] = get_ref_pose_single_bone_comp_space_transform(refSkeleton, i);
	}
	
	return outResult;
}




// Called when the game starts or when spawned
void AMyActor::BeginPlay()
{
    Super::BeginPlay();
    
    if (MySkel == nullptr)
        return;
    
    
    FTransform actorWorldTransform = this->GetActorTransform();

    // UE puppet's bone names:   
    TArray<FName> BoneNames = {
        "pelvis",
        "spine_01",
        "spine_02",
        "spine_03",
        "spine_04",
        "spine_05",
        "clavicle_l",
        "upperarm_l",
        "lowerarm_l",
        "hand_l",
        "index_01_l",
        "index_02_l",
        "index_03_l"
    };
    
    const FReferenceSkeleton& RefSkel = MySkel->GetSkinnedAsset()->GetRefSkeleton();
    
    for (FName name : BoneNames) {
        FTransform tr = GetRefPoseBoneTransforms(MySkel, name);
        
        FRotator EulerRot = tr.GetRotation().Rotator();
        FVector Loc = tr.GetLocation();
        DrawDebugCoordinateSystem(GetWorld(), Loc, EulerRot, 40.f, true);
        DrawDebugPoint(GetWorld(), Loc, 25, FColor(52, 220, 239), true);        
        
        
        tr = get_ref_pose_single_bone_comp_space_transform(RefSkel, RefSkel.FindBoneIndex(name)) * actorWorldTransform;
        
        EulerRot = tr.GetRotation().Rotator();
        Loc = tr.GetLocation();
        DrawDebugCoordinateSystem(GetWorld(), Loc, EulerRot, 40.f, true); 
        DrawDebugPoint(GetWorld(), Loc, 25, FColor(220, 52, 239), true);
    }
}

// Called every frame
void AMyActor::Tick(float DeltaTime)
{
	Super::Tick(DeltaTime);
}

Alternative way to find the bind pose

You can find the inverse of the bind pose directly in the USkeletalMesh as well: 

// This matrix is in component (object) space 
const TArray<FMatrix44f>& USkeletalMesh::GetRefBasesInvMatrix() const

// Updating the bind pose if something changed:
void USkeletalMesh::CalculateInvRefMatrices()

// When returned value is not null should be used instead of the above 
// (at least that what happens in UpdatePreviousRefToLocalMatrices())
USkinnedMeshComponent::GetRefPoseOverride();

Skin matrices (lbsReferenceToLocal)

A "skin matrix" is a matrix that takes a vertex in global position and transforms it to its animated/deformed position. In unreal engine skin matrices are often designated under the name ReferenceToLocal. You can get skin matrices *of the active bones* as follows in c++: 

TArray<FMatrix44f> lbsReferenceToLocal;
int32 lodIndex;

SkeletalMeshComponent::GetCurrentRefToLocalMatrices(
    lbsReferenceToLocal,
    lodIndex
);

Each element of lbsReferenceToLocal[] contains:
\[ W . B^{-1} \] where:

  • \( W \) is the joint's position (world space a.k.a joint's global transform)
  • \( B \) is the joint's bind matrix (reference matrix): world space transformation when bound to the T-pose.
Note: to get the joint's global transformation from a skin matrix you would need to pre-multiply by the global bind matrix (i.e. bone's global transformation in rest pose): \[ W = W . B{^-1} . B \]

Warning: GetCurrentRefToLocalMatrices() will return a list of transform which size matches the total number of joints, however, in some cases, it may not update some of the entries. For instance, if the bone is set to be visible, or it has no influence over the skeletal mesh. (Other parameters may influence as well such as if the bone is listed in FSkeletalMeshLODRenderData::ActiveBoneIndices;, FSkeletalMeshLODRenderData::RequiredBones)

Consequently, if you need to compute the skinning matrices of disabled bones, you need to force the update by calling directly the utility function UpdateRefToLocalMatrices(), and make sure to specify the list of ExtraRequiredBoneIndices: 

// SkeletalRender.h
// Unfortunately the header is in a private section:
// \Engine\Source\Runtime\Engine\Private\SkeletalRender.h
// So you need to copy paste the prototype of the function inside your cpp to access it:

/**
* Utility function that fills in the array of ref-pose to local-space matrices using 
* the mesh component's updated space bases
* @param	ReferenceToLocal - matrices to update
* @param	SkeletalMeshComponent - mesh primitive with updated bone matrices
* @param	InSkeletalMeshRenderData - resource for which to compute RefToLocal matrices
* @param	LODIndex - each LOD has its own mapping of bones to update
* @param	ExtraRequiredBoneIndices - any extra bones apart from those active in the LOD that we'd like to update
*/
ENGINE_API void UpdateRefToLocalMatrices( 
                    TArray<FMatrix44f>& ReferenceToLocal, 
                    const USkinnedMeshComponent* InMeshComponent, 
                    const FSkeletalMeshRenderData* InSkeletalMeshRenderData, 
                    int32 LODIndex, 
                    const TArray<FBoneIndexType>* ExtraRequiredBoneIndices=NULL );

Execution paths / implementation

Skinning is implemented 3 ways on:

  • CPU: designated as CPUSkin in UE
  • vertex shader (GPU): designated as Skin vertex factory in UE
  • compute shader (GPU): designated as skin cache in UE

Mainly located in:

  • CPUSkin: SkeletalRenderCPUSkin.cpp
  • Skin vertex factory: GpuSkinVertexFactory.ush GPUSkinVertexFactory.cpp
  • skin cache: GPUSkinCache.cpp

More official doc about skeletal mesh render paths

Skin Weights

Warning: the indices of the skin weights don't correspond to the global array ReferenceToLocalMatrices instead they are "local indices" to access the sub list of matrices ChunkMatrices[] that correponds to each mesh section. (this is true for most buffer that contains skin weights, wether on GPU or CPU, it seems UE converts the indices very early on when importing the asset. I'm not even sure the "global indices" are stored somewhere, that remains to be verified)

Just like matrices, skin weights get "chunked" into mesh sections. 


SkeletalMesh   
    LOD[0]    
    LOD[1]
    LOD[2]
        VertexBuffer[nb_vertices]
        // RefToLocal = WorldJointMat . WorldBindPoseMat^-1 = SkinningMatrixToApplyToVertices
        ReferenceToLocalMatrices[nb_joints_for_the_entire_skeleton] 

        MeshChunk[0] // MeshSection 0
            ChunkMatrices[nb_joints_influencing_the_section]
        MeshChunk[1] // MeshSection 1
            ChunkMatrices[nb_joints_influencing_the_section]

TODO: find out if there is a global buffer of skin weights stored somewhere (as opposed to the chunked ones)

Display skin weights

In the skeletal Mesh Editor:
Character -> Mesh -> Selected Bone Weight
Will show the skinning weights of the curently selected bone: (this will switch skinning computation to CPU and might slow down things)

Skin Weights Profiles

You can add extra set of skin weights (SkinWeightProfilesData etc.) to a skeletal mesh. In the Skeletal Mesh Editor you can add skin weights profiles in the details pannel:

In the viewport of the skeletal mesh editor you can switch between profiles: 

// \Engine\Source\Editor\Persona\Private\SAnimationEditorViewport.cpp
void SAnimationEditorViewportTabBody::Construct(...)
{

    ...

    .OnSelectionChanged(SNameComboBox::FOnNameSelectionChanged::CreateLambda(
    [WeakScenePtr = PreviewScenePtr](
        TSharedPtr<FName> SelectedProfile, 
        ESelectInfo::Type SelectInfo)
    {
        // Apply the skin weight profile to the component, according to the selected the name, 
        if (WeakScenePtr.IsValid() && SelectedProfile.IsValid())
        {
            UDebugSkelMeshComponent* MeshComponent = 
                WeakScenePtr.Pin()->GetPreviewMeshComponent();

            if (MeshComponent)
            {
                MeshComponent->ClearSkinWeightProfile();
                if (*SelectedProfile != NAME_None)
                {
                    MeshComponent->SetSkinWeightProfile(*SelectedProfile);
                }					
            }
        }
    }));

    ...
}

One can force a specific skin weight profile at runtime via blueprint:

Listing the names of the skin weight profiles: 

TArray<TSharedPtr<FName>> SkinWeightProfileNames;
USkeletalMesh* Mesh = ...;
{
    for (const FSkinWeightProfileInfo& Profile : Mesh->GetSkinWeightProfiles())
    {
        SkinWeightProfileNames.AddUnique(MakeShared<FName>(Profile.Name));
    }
}

Typical way to change the current skin weight profile (won't effect the deformer graph though): 

MeshComponent->ClearSkinWeightProfile();
MeshComponent->SetSkinWeightProfile(*SelectedProfile);

/*
TODO: check this change is persitent in standalone/packaged game.
e.g. create an AActor with skel mesh comp and see how it behaves.
*/

Non exhausitive overview of the data and methods related to skin weight profiles: 

// UE 4.26 – 5.1
USkinnedMeshComponent
 
    // Note: all methods below are
    // BlueprintCallable
    // ------------------
    // SkinWeightOverride

    // Allow override of skin weights on a per-component basis. 
    void SetSkinWeightOverride(int32 LODIndex, 
                               const TArray<FSkelMeshSkinWeightInfo>& SkinWeights);

    // Clear any applied skin weight override 
    void ClearSkinWeightOverride(int32 LODIndex);

    // Queues an update of the Skin Weight Buffer used by the current MeshObject
    void UpdateSkinWeightOverrideBuffer();

    // ------------------
    // SkinWeightProfile
    
    // Setup an override Skin Weight Profile for this component 
    bool SetSkinWeightProfile(FName InProfileName);
    
    // Clear the Skin Weight Profile from this component, in case it is set 
    void ClearSkinWeightProfile();
    
    // Unload a Skin Weight Profile's skin weight buffer (if created) 
    void UnloadSkinWeightProfile(FName InProfileName);
    
    // Return the name of the Skin Weight Profile that is currently set otherwise 'None' 
    FName GetCurrentSkinWeightProfileName() const;
    
    // Check whether or not a Skin Weight Profile is currently set 
    bool IsUsingSkinWeightProfile() const;
    
    // Check whether or not a Skin Weight Profile is currently pending load / create 
    bool IsSkinWeightProfilePending() const;     
    
    // Returns skin weight vertex buffer to use for specific LOD (will look at override) 
    FSkinWeightVertexBuffer* GetSkinWeightBuffer(int32 LODIndex) const;


    
    FSkelMeshComponentLODInfo LODInfo[];
        /** Vertex buffer used to override skin weights */
        FSkinWeightVertexBuffer* OverrideSkinWeights;
        /** Vertex buffer used to override skin weights from one of the profiles */
        FSkinWeightVertexBuffer* OverrideProfileSkinWeights;
    // END FSkelMeshComponentLODInfo -------



    /** 
        Object responsible for sending bone transforms, 
        morph target state etc. to render thread. 

        This abstract interface is implemented by 
        - FSkeletalMeshObjectGPUSkin 
        or 
        - FSkeletalMeshObjectCPUSkin
    */    
    FSkeletalMeshObject* MeshObject
        /**
        Will force re-evaluating which Skin Weight buffer should be used for skinning, 
        determined by checking for any override weights or a skin weight profile being set.
        This prevents re-creating the vertex factories, but rather updates the bindings in place. 
        */
        virtual void UpdateSkinWeightBuffer(USkinnedMeshComponent* InMeshComponent);

        /** Get the weight buffer either from the component LOD info or
            the skeletal mesh LOD render data 
            RODO: is this the same buffer as 
            FSkeletalMeshLODRenderData::GetSkinWeightVertexBuffer() ?
        */
        static FSkinWeightVertexBuffer* GetSkinWeightVertexBuffer(
                                            FSkeletalMeshLODRenderData& LODData, 
                                            FSkelMeshComponentLODInfo* CompLODInfo);
        
        /** Render data for each LOD */
        TArray<FSkeletalMeshObjectLOD> LODs;
            // UE: Skin weight buffer to use, could be from asset or component override 
            // ROD: This is the current skin weight buffer that will be used in:
            // - CPU skinning 
            //    (confirmed after analysing
            //     SkeletalRenderCPUSkin.cpp :: static void SkinVertexSection())
            //
            // - GPU Skinning (to be confirmed)
            // This is not used by the deformer graph which relies on 
            // data from FSkeletalMeshLODRenderData (see below)
            FSkinWeightVertexBuffer* MeshObjectWeightBuffer;
            void UpdateSkinWeights(FSkelMeshComponentLODInfo* CompLODInfo);

    // END FSkeletalMeshObject* MeshObject --------------

    USkeletalMesh
        const TArray<FSkinWeightProfileInfo>& GetSkinWeightProfiles() const
        void SetSkinWeightProfilesData(int32 LODIndex, 
                                       FSkinWeightProfilesData& SkinWeightProfilesData)
        TArray<FSkinWeightProfileInfo>& GetSkinWeightProfiles()
        void AddSkinWeightProfile(const FSkinWeightProfileInfo& Profile);
        int32 GetNumSkinWeightProfiles() const 
        void ReleaseSkinWeightProfileResources();


        FSkeletalMeshModel* ImportedModel;
            FSkeletalMeshLODModel LODModels[]
                FImportedSkinWeightProfileData SkinWeightProfiles[]; 
        
                // (Rendering/SkeletalMeshLODModel.h)
                FSkelMeshSourceSectionUserData UserSectionsData[]                 
                    ...
    
                // (Rendering/SkeletalMeshLODModel.h) 
                FSkelMeshSection Sections[] 
                    //Index in the TMap: FSkeletalMeshLODModel::UserSectionsData
                    int32 OriginalDataSectionIndex 
                    // CPU Skin weights, vertex position etc. 
                    // local (chunked) indices)
                    // Always holds the main skin weight profile.
                    FSoftSkinVertex SoftVertices[]; 
                    ...
       
        FSkeletalMeshRenderData* SkeletalMeshRenderData;
            FSkeletalMeshLODRenderData LODRenderData[];
                // Deformer graph will rely on this buffer:
                FSkinWeightVertexBuffer* GetSkinWeightVertexBuffer()
    
                // Skin weight profile data structures, 
                // can contain multiple profiles and their runtime FSkinWeightVertexBuffer 
                // ROD: you can make the "Skeleton" node of the deformer graph select and use
                // a specific profile using this:
                /*
                const FSkinWeightVertexBuffer* WeightBuffer = nullptr;		
                if (LodRenderData->SkinWeightProfilesData.ContainsProfile(ProfileName)) {
                    WeightBuffer = 
                        LodRenderData->SkinWeightProfilesData.GetOverrideBuffer(ProfileName);
                } else {
                    WeightBuffer = LodRenderData->GetSkinWeightVertexBuffer();
                }
                */
                FSkinWeightProfilesData SkinWeightProfilesData;
                    TMap<FName, FSkinWeightVertexBuffer*> ProfileNameToBuffer;
                    TMap<FName, FRuntimeSkinWeightProfileData> OverrideData;
                    ...
               

                           
                FSkelMeshRenderSection RenderSections[];  
                    // List bones used by this mesh section.
                    // returns *global* index 
                    // (i.e bones in the USkeletalMesh::RefSkeleton array
                    //  or also RefToLocalMatrices[] )
                    uint16 BoneMap[/* local (i.e same as chunkMatrix[]) bone index */];

CPU skinning

As far as I know cpu skinning is only used in specific scenarios in the editor and not the game runtime. It mainly gets enabled when displaying the skeletal mesh normals or tangent vectors or skinning weights. For instance you can force it through the menus:

more at "force cpu skinning.docx"

Note: on the CPU side skinning is also computed in many other places:/ A non exhaustive list of duplicated CPU skinning: 

USkinnedMeshComponent::ComputeSkinnedPositions()
USkinnedMeshComponent::ComputeSkinnedTangentBasis()
/** Simple, CPU evaluation of a vertex's skinned position helper function */
template <bool bCachedMatrices>
FVector GetTypedSkinnedVertexPosition()
/** Simple, CPU evaluation of a vertex's skinned position helper function */
void GetTypedSkinnedTangentBasis()
ClothingMeshUtils::SkinPhysicsMesh(…)

GPU skinning

Upload of skinning matrices to GPU

Skinning matrices are uploaded for Compute Shader (ue skin cache) and Vertex Shader (ue vertex factory) via the same procedure UpdateBoneData(), in other words, they use the same buffer. Now recall how skin matrices are organized in memory: 


SkeletalMesh   
    LOD[0]    
    LOD[1]
    LOD[2]
        VertexBuffer[nb_vertices]
        // RefToLocal = WorldJointMat . WorldBindPoseMat^-1 = SkinningMatrixToApplyToVertices
        ReferenceToLocalMatrices[nb_joints_for_the_entire_skeleton] 

        MeshChunk[0] // MeshSection 0
            ChunkMatrices[nb_joints_influencing_the_section]
        MeshChunk[1] // MeshSection 1
            ChunkMatrices[nb_joints_influencing_the_section]
The list of joints used by a particular mesh section is defined by TArray BoneMap; in: 
USkeletalMesh
    FSkeletalMeshRenderData* SkeletalMeshRenderData; // (SkeletalMeshRenderData.h) 
        FSkeletalMeshLODRenderData LODRenderData[];  // (SkeletalMeshLODRenderData.h)
            // FSkinWeightVertexBuffer* GetSkinWeightVertexBuffer()
            FSkinWeightVertexBuffer SkinWeightVertexBuffer;
            // Skin weight profile data structures, 
            // can contain multiple profiles and their runtime FSkinWeightVertexBuffer 
            FSkinWeightProfilesData SkinWeightProfilesData;
                       
            FSkelMeshRenderSection RenderSections[]; // (SkeletalMeshLODRenderData.h)
                // The bones which are used by the vertices of this section. 
                // Indices of bones in the USkeletalMesh::RefSkeleton array 
                uint16 BoneMap[];

UpdateBoneData() simply upload the sublist of matrices of each mesh section: 

// Engine\Source\Runtime\Engine\Private\GPUSkinVertexFactory.cpp

bool FGPUBaseSkinVertexFactory::FShaderDataType::UpdateBoneData() {
    // takes CPU arrays RefTolocal and upload to GPU
    // to Bufffer<float> BoneMatrix in the shader 
    // equivalent of glMap() (i.e. lockBuffer) is used for the upload.
    
    // GPU buffer are in a pool 
    // see GetBoneBufferForReading() GetBoneBufferForWriting() (see below)

    // Don't forget that we only upload matrices related to a mesh section.
    // and not every matrices of the USKeletalMesh.
}

class FGPUBaseSkinVertexFactory : public FvertexFactory {
    struct FShaderDataType {
        const FVertexBufferAndSRV& GetBoneBufferForReading(bool bPrevious, uint32 FrameNumber) const 
        FVertexBufferAndSRV& GetBoneBufferForWriting(uint32 FrameNumber) { 
            return BoneBuffer[alternate(FrameNumber)] 
        }

    private:
        // double buffered bone positions+orientations 
        // to support normal rendering and velocity (new-old position) rendering
        FVertexBufferAndSRV  BoneBuffer[2];

Skin Weights

Unreal encodes skin weights to 8 bits (0-255), this allows to send 4 skin weights via a single integer and compress data. Although maybe not waranted with modern GPUs, it was a way to avoid memory transfer bottle neck from CPU to GPU memory. 

// Example of unpacking uint8 skin weights to floats:
uint PackedBlendWeights = skinWeightBuffer[...];
float4 UnpackedBlendWeights = 0;
UnpackedBlendWeights.x = float(PackedBlendWeights & 0xff) / 255.0f;
UnpackedBlendWeights.y = float(PackedBlendWeights >> 8 & 0xff) / 255.0f;
UnpackedBlendWeights.z = float(PackedBlendWeights >> 16 & 0xff) / 255.0f;
UnpackedBlendWeights.w = float(PackedBlendWeights >> 24 & 0xff) / 255.0f;
return UnpackedBlendWeights;

Similarly if the total number of bones (for a particular mesh section) is under 255 then the indices get encoded to 8 bits 

int4 UnpackedBlendIndices = 0;

uint PackedBlendIndices = skinWeightBuffer[...];
UnpackedBlendIndices.x = PackedBlendIndices & 0xff;
UnpackedBlendIndices.y = PackedBlendIndices >> 8 & 0xff;
UnpackedBlendIndices.z = PackedBlendIndices >> 16 & 0xff;
UnpackedBlendIndices.w = PackedBlendIndices >> 24 & 0xff;

To handle higher number of bones Unreal also handles packing to 16 bits: 

int4 UnpackedBlendIndices = 0;

uint PackedBlendIndices = skinWeightBuffer[... + 0];
UnpackedBlendIndices.x = PackedBlendIndices & 0xffff;
UnpackedBlendIndices.y = PackedBlendIndices >> 16 & 0xffff;

PackedBlendIndices = skinWeightBuffer[... + 1];
UnpackedBlendIndices.z = PackedBlendIndices & 0xffff;
UnpackedBlendIndices.w = PackedBlendIndices >> 16 & 0xffff;

The “Skin Cache” a.k.a Compute shader

In the compute shader (GpuSkinCacheComputeShader.usf) we can access the skinning matrices. They will be blended with skin weights and applied directly to the vertices: 

Buffer<float4> BoneMatrices;

It is structured as follows: 

// BoneMatrices[boneIndex * 3 + 0] == first row
// BoneMatrices[boneIndex * 3 + 1] == second row
// BoneMatrices[boneIndex * 3 + 2] == third row
// No fourth row since in affine transformation it's alway (0 0 0 1)


Adding define to the compute shader (GPUSkinCache.cpp)
static void TGPUSkinCacheCS::ModifyCompilationEnvironment()

Compute shader dispatch:(GPUSkinCache)
void FGPUSkinCache::DispatchUpdateSkinning()

Enable compute shader over vertex shader in the Editor's console:

> r.SkinCache.CompileShaders 1

The “Vertex factory” a.k.a Vertex shader

(GpuSkinVertexFactory.ush) because of motion blur we need to compute skinning for current and previous frame in the vertex shader, so we find the bone matrices as follows (same structure as CS version): 

// The buffer of bone matrices (4x3) is stored as 3 ‘float4’ behind each other, 
// all chunks of a skeletal mesh in one, each skeletal mesh has it's own buffer
Buffer<float4> BoneMatrices;
// The previous bone matrix buffer (same format as above)
Buffer<float4> PreviousBoneMatrices;


// Adding defines to the vertex shader (skin vertex factory) GPUSkinVertexFactory.cpp:
void TGPUSkinVertexFactory<…>::ModifyCompilationEnvironment(…, FShaderCompilerEnvironment& OutEnvironment )
{
    FVertexFactory::ModifyCompilationEnvironment(Platform, Material, OutEnvironment);	
    OutEnvironment.SetDefine(TEXT("USE_DQS"),1.0); 
}

Sample code to compute skinning/LBS on CPU

FSkelMeshSection (imported data)

This version does not allow to switch between skin weight profiles. 

// MyActor.h
// Fill out your copyright notice in the Description page of Project Settings.

#pragma once

#include "CoreMinimal.h"
#include "GameFramework/Actor.h"
#include "MyActor.generated.h"

UCLASS()
class BONEMATRICESTEST_API AMyActor : public AActor
{
	GENERATED_BODY()
	
public:	
	// Sets default values for this actor's properties
	AMyActor();

	UPROPERTY(EditAnywhere);
	USkeletalMeshComponent* m_mySkel;

protected:
	// Called when the game starts or when spawned
	virtual void BeginPlay() override;

public:	
	// Called every frame
	virtual void Tick(float DeltaTime) override;

};

// ------------
// MyActor.cpp
// Fill out your copyright notice in the Description page of Project Settings.

#include "MyActor.h"
#include "DrawDebugHelpers.h"
#include "SkeletalRenderPublic.h"
#include "Rendering/SkeletalMeshModel.h"

// Sets default values
AMyActor::AMyActor()
{
    // Set this actor to call Tick() every frame.  You can turn this off to improve performance if you don't need it.
    PrimaryActorTick.bCanEverTick = true;

}

// Called when the game starts or when spawned
void AMyActor::BeginPlay()
{
    Super::BeginPlay();
}

// Called every frame
void AMyActor::Tick(float DeltaTime)
{
    Super::Tick(DeltaTime);
    if (m_mySkel == nullptr)
        return;
     
    FlushPersistentDebugLines(GetWorld());
     
    FTransform actorWorldTransform = this->GetActorTransform();
    FTransform componentTransform = m_mySkel->GetComponentTransform();
     
    const FSkeletalMeshObject* skelMeshObject = m_mySkel->MeshObject;
     
    USkeletalMesh* SkeletalMeshPtr = m_mySkel->GetSkeletalMeshAsset();
    FSkeletalMeshModel* SourceModel = SkeletalMeshPtr->GetImportedModel();
    // Get Active LOD
    const int32 lodIndex = skelMeshObject->GetLOD();
     
    // Or look up every LODs:
    //for (int32 lodIndex = 0; lodIndex < SourceModel->LODModels.Num(); ++lodIndex)
    {
        const FSkeletalMeshLODModel& lod = SourceModel->LODModels[lodIndex];
     
        //const TMap<FName, FImportedSkinWeightProfileData>& profiles = lod.SkinWeightProfiles;
        //UE_LOG(LogTemp, Warning, TEXT("Nb skin weights profiles: %d"), profiles.Num());
     
        TArray<FMatrix44f> lbsReferenceToLocal;
        m_mySkel->GetCurrentRefToLocalMatrices(lbsReferenceToLocal, lodIndex);
     
        uint32 numSections = lod.Sections.Num();
        for (uint32 sectionsIndex = 0; sectionsIndex < numSections; ++sectionsIndex)
        {
            FSkelMeshSection& section = SourceModel->LODModels[lodIndex].Sections[sectionsIndex];
             
            int32 nbVerts = section.NumVertices;
            for (int32 vix = 0; vix < nbVerts; ++vix)
            {
                const FVector3f& vertexPositionLocal = section.SoftVertices[vix].Position;
                const FSoftSkinVertex& softVert = section.SoftVertices[vix];           
 
                FMatrix44f mat(ForceInitToZero);
                for (int32 InfluenceIdx = 0; InfluenceIdx < MAX_TOTAL_INFLUENCES; InfluenceIdx++)
                {                       
                    uint32 localBoneIdx = softVert.InfluenceBones[InfluenceIdx];
                    uint32 boneIdx = section.BoneMap[localBoneIdx];
                    // Here weights are encoded to int16 (float should be between [0.0 - 1.0]
                    float weight = float(softVert.InfluenceWeights[InfluenceIdx]) / float(0xFFFF);
                    if (lbsReferenceToLocal.IsValidIndex(boneIdx)) {
                        mat += lbsReferenceToLocal[boneIdx] * weight;
                    }
                    else
                    {
                        UE_LOG(LogTemp, Warning, TEXT("Invalid index: %d"), boneIdx);
                    }
                }
             
                const FVector3f position = mat.TransformPosition(vertexPositionLocal);
				
#if 0
                const FVector& worldPosition =
                        actorWorldTransform.TransformPosition(FVector(position));
#endif			
				
                const FVector& worldPosition =  
                    componentTransform.TransformPosition(FVector(position));
 
                //const FVector& worldPosition = FVector(position);

                DrawDebugPoint(GetWorld(),
                               worldPosition,
                               10,
                               FColor(239, 220, 52),
                               true);              
            }
        }
    }
}


FSkeletalMeshRenderData (render data)

This version allows switching between skin weight profiles 

// MyActor.h
// Fill out your copyright notice in the Description page of Project Settings.
 
#pragma once
 
#include "CoreMinimal.h"
#include "GameFramework/Actor.h"
#include "MyActor.generated.h"
 
UCLASS()
class /*BONEMATRICESTEST_API*/ AMyActor : public AActor
{
    GENERATED_BODY()
     
public:
    // Sets default values for this actor's properties
    AMyActor();
 
    UPROPERTY(EditAnywhere);
    USkeletalMeshComponent* m_mySkel;

    UPROPERTY(EditAnywhere);
    FName SkinWeightProfile = NAME_None;

    UFUNCTION(BlueprintCallable, CallInEditor)
    void myFun();

 
protected:
    // Called when the game starts or when spawned
    virtual void BeginPlay() override;
 
public:
    // Called every frame
    virtual void Tick(float DeltaTime) override;
 
};


// MyActor.cpp
// Fill out your copyright notice in the Description page of Project Settings.
 
 
#include "MyActor.h"
#include "DrawDebugHelpers.h"
#include "SkeletalRenderPublic.h"
#include "Rendering/SkeletalMeshModel.h"
#include "Rendering/SkeletalMeshRenderData.h"
#include "Misc/AssertionMacros.h"

void AMyActor::myFun() {
    UE_LOG(LogTemp, Error, TEXT("my fun callback"));

    if (m_mySkel == nullptr)
        return;

    if (SkinWeightProfile != NAME_None)
    {
        bool exists = m_mySkel->GetSkinnedAsset()->GetSkinWeightProfiles().ContainsByPredicate(
            [this](const FSkinWeightProfileInfo& Profile) {
                return Profile.Name == SkinWeightProfile;
            }
        );

        if (exists) {
            UE_LOG(LogTemp, Error, TEXT("Set skin weight profile"));
            m_mySkel->ClearSkinWeightProfile();
            m_mySkel->SetSkinWeightProfile(SkinWeightProfile);
            m_mySkel->UpdateSkinWeightOverrideBuffer();
        }
        else {
            UE_LOG(LogTemp, Error, TEXT("skin weight profile not found"));
        }
    }
}

// Sets default values
AMyActor::AMyActor()
{
    // Set this actor to call Tick() every frame.  
    // You can turn this off to improve performance if you don't need it.
    PrimaryActorTick.bCanEverTick = true;
 
}
 
// Called when the game starts or when spawned
void AMyActor::BeginPlay()
{
    Super::BeginPlay();
}
 
// Called every frame
void AMyActor::Tick(float DeltaTime)
{
    Super::Tick(DeltaTime);
    if (m_mySkel == nullptr)
        return;
     
    FlushPersistentDebugLines(GetWorld());
     
    FTransform actorWorldTransform = this->GetActorTransform();
    FTransform componentTransform = m_mySkel->GetComponentTransform();
    
    const FSkeletalMeshObject* skelMeshObject = m_mySkel->MeshObject;
    USkeletalMesh* SkeletalMeshPtr = m_mySkel->GetSkeletalMeshAsset();

    // Get Active LOD
    const int32 lodIndex = skelMeshObject->GetLOD();
     
    const FSkeletalMeshRenderData* renderData = SkeletalMeshPtr->GetResourceForRendering();

    // Or look up every LODs:
    //for (int32 lodIndex = 0; lodIndex < renderData.LODRenderData.Num(); ++lodIndex)
    {
        
        const FSkeletalMeshLODRenderData* lodRenderData = &(renderData->LODRenderData[lodIndex]);

        const FSkinWeightVertexBuffer* weightBuff = 
            lodRenderData->GetSkinWeightVertexBuffer();        

        if (lodRenderData->SkinWeightProfilesData.ContainsProfile(SkinWeightProfile)) {
            weightBuff = 
                lodRenderData->SkinWeightProfilesData.GetOverrideBuffer(SkinWeightProfile);

            if (weightBuff == nullptr) {
                UE_LOG(LogTemp, 
                       Error, 
                       TEXT("null weightBuffer for profile name: %s; switch back to standard"), 
                       *(SkinWeightProfile.ToString()));                
            }
        }

        check(weightBuff != nullptr);

        UE_LOG(LogTemp, Warning, TEXT("needs CPU access: %i"), weightBuff->GetNeedsCPUAccess());
      
        TArray<FMatrix44f> lbsReferenceToLocal;
        m_mySkel->GetCurrentRefToLocalMatrices(lbsReferenceToLocal, lodIndex);

        const FPositionVertexBuffer& vertexBuffer =
            lodRenderData->StaticVertexBuffers.PositionVertexBuffer;
        
        int32 nbVerts = lodRenderData->GetNumVertices();
        for (int32 vix = 0; vix < nbVerts; ++vix)
        {
            const FVector3f& vertexPositionLocal =
                vertexBuffer.VertexPosition(vix);

            int32 sectionIdx, localVertIdx;
            lodRenderData->GetSectionFromVertexIndex(vix, sectionIdx, localVertIdx);
            const FSkelMeshRenderSection& section = lodRenderData->RenderSections[sectionIdx];

            FMatrix44f mat(ForceInitToZero);
            for (int32 InfluenceIdx = 0; InfluenceIdx < MAX_TOTAL_INFLUENCES; InfluenceIdx++)
            {

                FBoneIndexType boneIdx = weightBuff->GetBoneIndex(vix, InfluenceIdx);
                uint16 w = weightBuff->GetBoneWeight(vix, InfluenceIdx);

                // 
                check(section.BoneMap.IsValidIndex(boneIdx));

                // Althoug we use the global skin weight buffer,
                // the joint indices are already expressed in local indices<br>
                // (section/chunkMatrix[]) ready to be uploaded on GPU.

                // So we need to convert back to global since we use lbsReferenceToLocal[]
                // in this sample code
                boneIdx = section.BoneMap[boneIdx];
                
                                                
                float weight = 0.0f;  
                if( weightBuff->Use16BitBoneWeight() )
                    // TODO: to be checked with a model that uses 16bit weights
                    weight = float(w) / float(0xFFFF); 
                else
                    // Data is actually coded to 8 bits
                    weight = float(w >> 8) / float(0xFF);

                check(lbsReferenceToLocal.IsValidIndex(boneIdx)) 
                mat += lbsReferenceToLocal[boneIdx] * weight;
                
            }

            const FVector3f position = mat.TransformPosition(vertexPositionLocal);

            FVector worldPosition =
                m_mySkel->GetComponentTransform().TransformPosition(FVector(position));
                //this->GetActorTransform().TransformPosition(vertexPositionLocal);

            DrawDebugPoint(GetWorld(),
                worldPosition,
                10,
                FColor(239, 220, 52),
                true);

            DrawDebugPoint(GetWorld(),
                FVector(position),
                10,
                FColor(52, 220, 239),
                true);

            DrawDebugPoint(GetWorld(),
                FVector(vertexPositionLocal),
                10,
                FColor(52, 239, 52),
                true);
        }

    }

}


FSkeletalMeshObject (render data)

As far as I can tell, these skin weights are used in the fixed pipeline (CPU and GPU) TODO: check if different from FSkeletalMeshRenderData skinWeightBuffer.

FSkeletalMeshLODModel (imported data)

TODO: code sample using: (question: are the bone indices already expressed in local (mesh section) indices?

USkeletalMesh::FSkeletalMeshModel::FSkeletalMeshLODModel::FImportedSkinWeightProfileData SkinWeightProfiles[];

Editor, UI, Slates

One can add a combo-box selector to allow the user to specify which existing Skeletal Mesh asset he wants to use:

If your class/Actor/UObject etc. is already part of a UI system, and associated for instance to a "detail" pannel, then thanks to introspection magic you will only need to add the UPROPERTY which will automatically add the corresponding combo box element in the your existing pannel/menu (creating the menu itself is much more involved I think) 

UPROPERTY(EditAnywhere, Category = Mesh)
TObjectPtr<USkinnedAsset> skinnedAsset;

UPROPERTY(EditAnywhere, Category = Mesh)
TObjectPtr<USkeletalMesh> skeletalMesh;

// Similarly:
UPROPERTY(EditAnywhere, Category = Mesh)
TObjectPtr<USkeletalMeshComponent> skeletalMeshComponent;

Loading process from FBX

At the beginning we call ImportSkeletalMesh() to create a USkeletalMesh and intermediate data ImportSkeletalMeshArgs: 

USkeletalMesh* UnFbx::FFbxImporter::ImportSkeletalMesh(FImportSkeletalMeshArgs& ImportSkeletalMeshArgs)

Fbx nodes will be read and converted to intermediate structs such as: FSkeletalMeshImportData* SkelMeshImportDataPtr This mostly happens in the Fillxxxxx() methods. finally intermediate data is converted to a USkeletalMesh through ProcessXxxxx() methods or at the root of ImportSkeletalMesh().

Vertex attributes

There seems to be facilities to handle weight maps (e.g. associate a float per vertex), or in other words, vertex attributes. I have no idea if the feature in the core of the engine is stable or in active development 


class FSkeletalMeshLODInfo
    /** List of vertex attributes to include for rendering and what type they should be */
    UPROPERTY(EditAnywhere, Category = SkeletalMeshLODInfo, AdvancedDisplay, EditFixedSize, Meta=(NoResetToDefault))
    TArray<FSkeletalMeshVertexAttributeInfo> VertexAttributes;	



class FSkeletalMeshLODRenderData 
    FSkeletalMeshVertexAttributeRenderData VertexAttributeBuffers;

For now you have to use the "Skeletal Mesh Editing Tools" (Experimental plugin when checking in 2023/08/02) that seems to take advantage of these facilities. In the skeletal mesh editor you can go and add vertex attributes (edit attributes) and then hand paint them (paint maps). I don't think there is a way to import or export the weight maps created inside the editor (yet?)

Attributes will show up the "details" pannel of the skeletal mesh editor:

You can then use those attributes inside the deformer graph (use the node "kinned Mesh vertex Attributes"

FBX vertex attributes

I'm not sure this is related to the above plugin but you can import vertex attributes via FBX:

Morph targets (a.k.a blend shapes)

WIP

Deformer graph

The deformer graph is a node system that allows to code your own compute shaders to deform a skeletal mesh.

One comment

Super awesome article, it helps me read UE source code more efficiently, I would appreciate it if you can finish “Morph targets” part!

annayin - 19/12/2023 -- 13:06
(optional field, I won't disclose or spam but it's necessary to notify you if I respond to your comment)
All html tags except <b> and <i> will be removed from your comment. You can make links by just typing the url or mail-address.
Anti-spam question: