RBF Pose Interpolator Math Help

Rigging
#1

The studio I work for is in need of an RBF pose interpolator for use in Maya facial rigs to drive control transforms. The existing pose interpolator in Maya 2017+ works fairly well (though other RBF interplation options would be nice), but outputs only the weight values for each pose. If possible, I’d like an option similar to BraveRabbit’s weight driver, which I unfortunately can not use due to studio policy.

Essentially, N driver attributes for the pose space to drive N target attributes.

I’ve started writing a custom python plugin node for this purpose, but I’m tripping up on the math required to make this work. For simplicity’s sake I’m using SciPy’s Rbf function, but I keep running into errors and I blame the fact that while I think I know what I’m doing, I probably don’t.

Has anyone tried authoring an interpolator successfully with the above requirements? Are there any useful resources for non-mathematicians? :slight_smile: Any help with to clarify or explain the applied math here would be greatly appreciated. Thanks!

3 Likes
#2

Hey Alan,

Here is one thing I have in regards to RBF math, I saved it and haven’t gotten around to reading it yet.

And in regards to the weight driver, there is an accompanying UI build around it and other rbf nodes.


(towards the bottom)
The video is a little long in the tooth, but it gets the point across.

Let us know if you get something (open source) working.

#3

Thanks Rafael!

I think the interface in that manager is definitely what would be desired once the driver is authored. It’s funny too that it’s intended to use the same weight driver, but it’s a dependency that is on the user to obtain. This is one of the reasons we don’t want to necessarily go down this route for future maintenance reasons.

I’ve been going through Marcus’sblog post on the subject as well. It’s good… but leaves an awful lot to assumption of familiarity with the mathematics at play.

He’s basically constructing a distance matrix for driver pose values. Constructing a second distance matrix of driven pose values, then solving Ax = B to get a ‘weight matrix.’

This is where I get lost. He solves for the interpolation point and gets some rotation numbers which are then applied against his weight matrix, but I’m not solid on how he got these rotation numbers. :frowning:

#4

I definitely agree about the ‘outside’ maintenance aspect. The RBF manager in that video is using a subclass of generic calls that with a little elbow grease you can support Maya’s rbf node and any proprietary ones you may author.

I feel your pain on the math, always a slow point for me.

#5

I am VERY confused by that blog post. The weight matrix he presents is not a linear solution for the values he gives.

#6

Something that tripped me when working with RBF and euclidean distance, is that in most function types, you need to pass the inverse coefficient (the smaller it is the larger the falloff).

In regards to that post, I’ll be honest I’m not sure what is being communicated. I’m assuming that he’s simply using linear algebra to solve his approximation.

Someone at the office supplied me with a machine learning video which not only helped with the math, but clarified how it applies conceptually.

Hopefully this gets you closer.

Note that in that video he’s using a Guassian radial function and he simplifies the equation. I used the wikipedia function and it works as intended e^-(B * distance)**2.

PS.: I’ve been having a hell of time solving multi-quadratic approximations… if anyone has more insight on this type of function I’d appreciate the help.

#7

Thank you to everyone who’s taken the time to reply to this thread. It’s all been very helpful!

I believe I have an algorithm that, while it does need more stress testing to examine the results, seems to give me what I’m looking for.

For those that want to check into this, here’s where I’m at:

The equation we need is image

If X in this equation is the driver values we’re trying to find the pose for, then there needs to be some function to run on those driver values that will give us our result. That function assumes we have a set of test data (our set poses) to work with, which in this case is represented by ‘k.’ So for every bit of test data, we need to find the distance between X (our driver values) and that test data. That answer will be run through some other function if desired (image) and then multiplied by some unknown weight value (‘wk’). All those answers will be added to together and then BAM… that’s the pose you want.

The problem is that we don’t know what that ‘wk’ is for each of our test data. That’s what Marcus appears to be trying to solve for initially using Ax = B.

To do this, I’m using scipy’s libraries to handle the heavy lifting. If you run every driver value in your data set through the pdist function to get the complete . I then squareform that result for use in Ax = B. The trick is to remember that we will likely run this distance through a basis function. Whatever function we choose, this distance matrix needs to have it applied prior to solving Ax = B. For reference on what some potential basis functions look like, check out scipy’s Rbf class. While I didn’t use that class specifically, the documentation reference is pretty handy. Also, thankfully, scipy’s functions allow for matrix inputs so if you want to find exponential results you can in one line (i.e. exp, sqrt, power, etc).

From here, scipy comes to the rescue again with solve. This will solve our Ax = B problem that we find in Marcus’ post. If we construct our desired answer, B, with a matrix of all the data sets’ pose values and our A is the result of the distance work we just did and pump it into this ‘solve’ function, we get our weight values (‘wk’). I will say that I’m not sure how Marcus came to that weight matrix he has. Scipy’s results are very different from both his as well as the matrix math site he points to. My only guess is that he is using some sort of basis function to get those numbers but doesn’t reveal which one :frowning:

All this information is developing our ‘pose space.’ We now have enough info to solve the RBF function above on our data set with, hopefully, any given test input.

So if X is whatever my input values are, I should now be able to find the distance between it and every driver value in my data set. I run that answer through the exact same basis function I used to solve for the weights and multiply by the given weight. Keep in mind that the weight value in this case is going to be an array of values. For example (if we for a moment just assume Marcus’ matrices are all correct):

image - image

is run through a basis function, then multiplied by image

We then do this for every row of starting driver values and add all those result up. Our answer is then the pose values we’re looking for.

I’ve been testing this by trying this process with one of the driver values in the initial data set (-1, 0, 1 for example). If I get the corresponding pose value in the data set as my result, I’m feeling pretty good that I’m doing alright. Part of the usefulness of the RBF approach is that if we’re interpolating through known data points we should be getting our known values.

That’s what I have for now. I’m successfully using a gaussian basis function, but I also am having an issue with multi-quadric and thin-plate functions. I wouldn’t be surprised if I’m missing something. Next as well is to use some sort of ‘decomposition’ to help with Ax = B when A and B are not of similar dimensions.

TLDR: image

1 Like
#8

Hey Alan,
How you doin?

This is a helpful project when it comes to the math and code side for this. It should have many of the functions you would need (except Thin Plate I believe).
Hope that helps.
Cheers,
-Sean

#9

Check out this lecture on RBFs from a Caltech machine learning course:


An RBF setup should be simple, you have a series of input values (poses) and a mataching series of output values (whatever needs to be driven), the falloff is defined by the basis function acting on a distance, which doesn’t necessarily have to be euclidean, and the interpolation is modified by the weights.
2 Likes
#10

Thanks for your outline A_Weider, it truly helped.

Btw, using the scipy.interpolate.Rbf class seems to be a tiny bit faster.

Quick example on how I got this working:

Assuming your data set of pose_drivers and pose_values are always 2 dimensional arrays with the same number of rows…

inputs = pose_drivers.T
rbfs = [scipy.interpolate.Rbf(*np.append(inputs, value_row).reshape(-1, inputs.shape[-1]), 
        function='gaussian') for value in pose_values.T]

to approximate your input value:

output = [rbf(*input) for rbf in rbfs]

Thanks again!

2 Likes