3dsMax + Kinect SDK, slowdown with multiple skeletons

Hi! I’m playing around with Microsoft Kinect SDK and 3dsMax, building a cheap mocap system.
It’s very simple to have some dummies moving in 3dsMax viewport, but I’m stuck on a performance issue sending to max the data for the two Kinect tracked skeletons.

This is my scenario:
I have a dotnet form showing the video and depth feed from kinect and a 2d screen space view of the tracked skeleton. When I send the data of the first (default) skeleton to 3dsMax, both the video feed and the 3d viewport run smoothly. When I send the data for the second skeleton too, the 3d viewport is still running at a good framerate, but the interface of both max and my dotnet form become inusable.

I’ve tried to send the data to 3dsMax using another thread, but it’s even worst, crashing all the time.

I’m suspecting that the problem is in the way I send the tracked data to max, using maxscript:

ManagedServices.MaxscriptSDK.ExecuteMaxscriptCommand("$dummy_" + prefix.ToString() + JointID.ShoulderLeft.ToString() + ".pos = [" + (skeleton.Joints[JointID.ShoulderLeft].Position.X * skeletonSizeMultiplier).ToString(NumberFormatInfo.InvariantInfo) +
                                                                                                             "," + (skeleton.Joints[JointID.ShoulderLeft].Position.Z * skeletonSizeMultiplier).ToString(NumberFormatInfo.InvariantInfo) +
                                                                                                             "," + (skeleton.Joints[JointID.ShoulderLeft].Position.Y * skeletonSizeMultiplier + skeletonZoffset).ToString(NumberFormatInfo.InvariantInfo) + "]");

the code above is the simplest way to move somthing in space, but seems to works…for one skeleton at least :slight_smile:

Any suggestion to improve the data transfer from c# to 3dsMAx?

thanks,
Alessandro

  1. Minor thing but use string formatting rather than concatenation (String.Format(’{0}.foo’, obj.Name) rather than ‘obj.Name + “.foo”’). Saves memory.
  2. If that’s the only thing you’re doing in script, I’d look at doing it through the SDK instead (look at the .NET sdk as well). Trying to run a script every frame and get interactive speeds is terribly difficult (I wrote an entire cloth sim and 90% of the time was spent updating the bone objects ($blah.transform = foo) and I could only get maybe 15 fps with one char).
  3. Max is inherently single-threaded, as you found out :slight_smile:
  4. If you don’t want to use the SDK, I don’t blame you- maybe try sticking the code in script controllers, rather than running it globally. Not sure if that’ll help perf.

Hi Rob, thanks for your advises.

The only thing I’m doing in maxscript is the data transfer to the dummy bones…consider that I was calculating in mxs, on every kinect skeleton update for 20 boxes, a simple distance (distance $A.pos $B.pos) and what I got was 10 fps…

It’s true, I’m trying to avoid the c++ maxSdk as possible by years…but:
about the .NET sdk, are you meaning the official one or the MAX.NET third-party wrapper?
I know from this post that the official .NET sdk will be out by the end of september…
Anyways I’ll try to put the transform code to a scripted controller and I’ll let you know if it works.
thanks again

just an update,
with the script controller the slowdown is the same as before.
Looking forward for .NET SDK solution…

Try this- in the script, do EVERYTHING except apply the transformation update to the 20 boxes. See what sort of speeds you’re getting.

Also there are some performance numbers for doing stuff in MXS on the wiki, check them out and you’ll see how awful MXS interaction performance is :slight_smile:

And yes the MAX.NET SDK was a 3rd party but AD is now including it, or bought it, or whatever. Same one I’m pretty sure.

I’ve commented the lenght update for the 20 boxes/skeleton, but the results are exactly the same: good fps in max viewport, but bad .net GUI interactivitity (around 5fps for the kinect video feedback).

thanks again Rob

Darn. Could it be due to to the overhead of receiving the IO for the additional skeleton? Have you looked at doing async IO (not sure that’d help), though who knows how Max would handle that (by crashing, my guess is).

How is your code communicating with Max/your .NET GUI? Is everything happening in-process?

Yes, the C# GUI - 3dsMax communication happens in process, when a new skeleton-tracked event is triggered, I run the MXS code to update my dummies.
I’ve tried to run the Maxscript code in a new thread, for async I/O, but everything become very instable.

You can still do async on a single thread. The point of async is to be non-blocking just as much as it is to take advantage of multithreading. So while IO is going on, your software thread which would otherwise be doing nothing while waiting for IO, can be doing actual work.

How big is the code? Can you post it? I can try to look it over for potential performance problems or ideas.

I could be wrong…but it seems that doing async calls in C# is very similar do operation on a new thread…
Am I missing something?

this is the code where I run the maxscript update (still over the first tracked skeleton) in async way: it crashes 3dsMax in the same way of multi-thread operations.


[...]
            //update the 3dsMax dummies
            if (m_passDataToMax)
            {
                SkeletonFrame allSkeletons = e.SkeletonFrame;
                //get the first tracked skeleton
                SkeletonData skeleton = (from s in allSkeletons.Skeletons
                                         where s.TrackingState == SkeletonTrackingState.Tracked
                                         select s).FirstOrDefault();

                //async IO
                DelegateWithParameters runMxsUpdate = new DelegateWithParameters(m_mxs.m_MXS_updateDummies);
                runMxsUpdate.BeginInvoke(skeleton, "skeletonA_", m_skeletonSizeMultiplier, m_skeletonZoffset, null, null);
            }
[...]

Hello again Rob,
the problem is solved now…without using any C# advanced feature :slight_smile:

I moved the code running the skeleton dummies update from C# gui to Maxscript event, and now it works perfectly.

I used something like this to run my C# method every 1/30 seconds:
(kinectInterface is my own .NET assembly)

theTimer = dotNetObject "System.Windows.Forms.Timer"
fn updateSkeletons = (kinectInterface.updateSkeletons())
dotnet.addEventHandler theTimer "tick" updateSkeletons 
theTimer.interval = 30
theTimer.start()

I wish to add that I’m using this kind of update only to have a realtime preview of the movements of the skeletons tracked by Kinect, because maxscript does NOT ensure me that the event is triggered every 1/30 seconds.
So when I need to create an animation, I store every frame in memory and then apply back to the skeleton in 3dsMax.

I’ll be happy to post some videos/screenshots when ready, if I get the permission to share the results.

Thank you for helping me,
Alessandro

Great news Alessandro! I should have guessed, as it is the complete opposite of how most optimizations go!

As for the async stuff, I have some blog posts that may help, here’s one:


And all my posts about threading:
http://www.robg3d.com/?tag=threading

Hi Alessandro,

I did something in the past using OpenNI, when the kinect SDK was not out yet. And i managed to get it running pretty fast. I remember running into the same problem you had and I found a bunch of tricks when sending stuff to max to process it.
I can’t really remember right now the code but I can have a look at it later and let you know if you are still interested.

BTW, say hi to Christian Ronchi :slight_smile:

Best,
Ruben.

Hi Ruben!
I was just talking about you some days ago regards your experience with kinect :slight_smile:
I’m writing a pm with the details.

I will say ciao to Christian as soon as possible!

ciao,
Alessandro