General

kemorig is a 3D VTuber software. It tracks your movements in realtime and outputs a video feed of a virtual avatar. It is intended for use while streaming live on platforms like twitch.tv.

Custom avatars and levels are authored in Unreal Editor and imported into kemorig at runtime.

kemorig is ideal for vtuber authors who want higher fidelity, more control, and easier iteration.

A killer feature is that content is exported from Unreal Editor as generic .pak files. This means your avatars and levels can contain virtually anything you can build in Unreal. (Plus, the skills you learn are directly transferable to Unreal gamedev.)

A specific goal of mine is to create vtubers that blend hand-authored animations and come to life like game characters. That, and to avoid the headaches and limitations of existing vtuber file formats.

A final price is undecided, but it will probably be a small one-time fee.

During Early Access, early builds are given to patrons on Patreon (Any builds you acquire while a patron are yours forever, even after your subscription ends).

Features

Currently supported:

  • iPhone (Live Link Face App) - ARKit Face Tracking and Head Rotation
  • Webcam - Face Tracking, Head Rotation and Position
  • Microphone (lip sync) - OVR Visemes

Planned for later releases:

  • More iPhone ARKit Apps
  • Leap Motion - hand tracking
  • Spout - high quality video with transparency.
  • Window Capture - transparency supported.

Yes, and it's very powerful! But you have to script them in Blueprint.

kemorig gives you a framework to make your own interactions. These can trigger animations for your character, or run arbitrary Blueprints to spawn props, throw things, manipulate the camera, and more. Interactions you make appear in the UI and have hotkey support.

Various types of interactions are supported:

  • One-Shot interactions
  • Toggle interactions
  • Randomly selected interactions
  • "Stance" interactions (a collection of states, with transitions, and you can be in one at a time)

In the future, I may add some pre-made interactions to set up common features people like to have on their streams.

Avatars

No. In order for kemorig to do what it does, it can't use standardized cross-app file formats like VRM.

If you have a VRM avatar you would like to use in kemorig, see the next question.

You will need to have your avatar model saved to a .fbx file. This is what you will import into Unreal, and from there you configure it in Unreal Editor for kemorig.

An fbx file is typically exported from a 3D Package (e.g. Blender, Maya). The model in a 3D Package is sometimes called the "source asset". If you don't have your avatar's source asset, and instead have something like a VRM file, you can generally find ways to import them into 3D Packages. From there you can export as fbx.

You can also find 3rd party plugins for importing VRM files into Unreal, although I have not tested them, and would recommend starting with a source asset in a 3D Package.

No. Your avatar can be anything you want, as long as it's a mesh that Unreal can import.

Certain tracking features (e.g. head rotation) must be configured in Unreal Editor for your avatar, but they are designed to accomodate any skeleton and bone orientations.

To animate your character, you'll need to author animations in a 3D package (e.g. Blender, Maya) and import animations into Unreal.

kemorig does not come with animations to apply to your character. There is no "default skeleton" or assumptions about your character's skeleton, and likewise it does not come with animations for your character's custom skeleton.

Note that Epic supports their own Mannequin skeleton on the Unreal Marketplace, which is a fairly generic biped skeleton. If you like, you can find models and animations on the Unreal Marketplace using the mannequin skeleton. You can use these directly in Unreal, or you can retarget animations to your own biped skeleton.

Also note that you can author static poses relatively easily in Unreal Editor, so if you have no animations at all you can still create a pose.

Technical Details

It uses Unreal's Forward Shading Renderer. This choice was made to support superior anti-aliasing (MSAA) and generally faster baseline performance.

It runs in DirectX 11 mode. This choice was made for better backward-compatibility, and because DirectX 12 features aren't critical for a vtuber app.

When running the kemorig app, performance will depend on the complexity of the character and scene. But in general, please refer to Epic's system requirements for Unreal Engine. Note that Epic lists Windows 10 as a minimum requirement, however kemorig uses DirectX 11 and may run on Windows 7 (although this is not tested).

Also note that running Unreal Editor (to author content) is generally more demanding than running the app itself.

No. Multi-platform support would be nice, but it is not happening in the forseeable future.