Anyone interested in contributing to an open source asset management system on github? I have been kicking around the idea lately of making a python/mysql asset management system in my spare time.
Anyone have experience with stuff like this already? Any pointers/feedback?
I’m interested, time permitting. Sounds like it could be fun. I co-developed the metadata-based asset management system we use, which is partlially Python-based.
Offhand I would recommend considering SQLAlchemy for the database backend. It has an amazing ORM layer, and keeps everything independent of the database implementation (MySql, MSSQL, Postgres, Oracle, etc.), among other tihngs.
I’m currently creating my own database program using python, I’d like to help as well. PM me with further info.
I’d be interested, would be a fun project. Done bits and pieces of implementations at various companies, would be cool to coalesce some of that.
yes!
There are a lot of asset management tools out there, but none targeted toward some of the game development issues. And any production related tools are always waterfall driven, which is not always how things go.
I think this sounds like a really good project I was planning on attempting something very similar for part of my Final Year Project.
Not sure if I can help, I’m not as skilled as everyone else on here (maybe one day lol) but I will certainly be interested in how it develops.
Can you describe what you mean by ‘Asset Management’? It is funny to hear things like “There are a lot of asset management tools out there, but none targeted toward some of the game development issues” when the term is so nebulous and poorly defined.
I’m also interested!
The following is a teeny bit offtopic but I’d like to give my current view on the asset manager issue so the more experienced among you can determine whether i could be of assistance or not.
My perspective comes from a commercial/vfx background but the similarities in rapport to games pipelines seem overwhelming.
I am currently developing python modules for 3dsmax-fusion-mySQL integration.
Heres some ramblings about what I’ve envisioned an asset manager needs to be for a small studio doing commercials and the occasional vfx shot:
The asset managers ive seen are usually horribly cluttered with all sorts of discouraging “features”
Personally i like the idea of contextual browsing of assets which means artists are required to work in specific folders e.g. /really_cool_project_5589/meta/max/workfiles/sh01/ and they select the session from a drop down list(in case there is say 2 maxes and a fusion runinng) whose assets the wish to work with.
Also there would be a way to make an the asset global so that it is listed on all shots.
Im thinking of using containers(.maxc) in max for defining assets.
Validation of the assests via scripts and above all common sense.
It needs to be extremely simple, as few ui elements as possible so that it engages the user more and requires them to make sure they’ve reached the definition of done.
Ideally there would be:
-A drop down list for the session and another for the version(or just use svn or something , as long as it can handle loads of data quickly)
-An elegant way of listing assets (perhaps even as nodes, familiar to compositors)
-just a few buttons like “merge asset” “save asset”, or better yet just drag and drop straight between manager/max/fusion… Currently I dont know how one goes about doing this though.
-Keep It Simple Stupid.
I can already pass cameras between max, fusion and mysqldb and im working on the listing of active sessions, and have found myself wishing for something else than COM for the job… Fusion has python natively so its not a problem. Things are looking better though with max atm the_maxsharp_project_python_and_c_from_3ds_max_2011
It’s all this simple due to the fact we have a certain way rendering(via custom plugin that does versioning) and doing things. Theres no software that can replace a tried&true method the artists like, unless its easy to learn and fixes exactly what they wanted fixed and no more. The only reason for completely revolutionizing a workflow would be scalability for managing hundreds instead of dozens of artists. This however is a topic i cannot contribute to as of yet.
Hope this made some sense.
Comments, Ideas, possible pitfalls, criticism and relocation to another thread are all welcome 
I guess, we do need more clarification. I guess I was reading it as “project management” i.e., issue tracking + asset tracking + work status, etc.
Most of the generalized tools (MSProject, Devtrack, etc) aren’t specifically geared toward the type of work we do in games. There are some good tools out there for film VFX like Shotgun.
I guess you guys are talking about something entirely different, (perforce/alienbrain), so I’ll politely back slowly out the front door.
there was one attempt at doing MEL based asset managment tool called openpipeline, it was fairly straight forward in concept but interface wasn’t that intuitive, but works well or small animation pipelines…
I remember it having separate section for saving assets and animation/layout/lighting. but no backend database connection as such…
To sum it up in a sentence or so:
An app that allows people to select an art asset from a database, select an assoc task to do, get feedback on it, complete it, have it be validated and checked in, while never having had to open perforce or antyhing.
I’ve been wanting to do something like this for work for a while now. From what you’re describing you want a project management/approval system with some hooks to an asset database.
The way I think of it, it’s best to have separate modules so if people already have devtrack/tfs/jira/bugzilla/trac for their task management, it should wrap itself around that. So on and so forth for each module. So you can throw a review module on top of it, or tie it with a revision control (or not). The review module could be replaced with an existing codereviewer or mixed together.
The desire to have it work in a way that would work in more than one environment is going to be interesting, become each project has a very different workflow.
I’ve messed with a lot of stuff like that, from building a website in python that speaks to our game databases, to distributed crawling/parsing scripts that grab attributes from maya/unreal files and dump them to a database and a ton of little stuff in the middle.
When you have thousands of assets, having something like that becomes really crucial. We have most of the parts but not one main skynet that links everything together, parts of it, but not everything.
I signed up for the shotgun beta. They said I should have an account set up by monday. Have any of you guys checked it out?
I’ve looked at shotgun before, and it’s really impressive, especially if you’re in a film production.
My only issues with it were that it wasn’t too game friendly from what I saw. The example game they had was for a football style game, and I worked on Madden/NCAA and could tell you that’s not how I would set it up =) Maybe the folks that are there now disagree with me.
The other issue was that it seemed like you had to switch to shotgun all the way, and in most studios that’s really hard to do. If your production team is used to using a piece of tools that they’re happy with, it’s really hard to convince them otherwise. But all of the asset management/task tracking/bug reporting were in one place, which is nice but hard to sell (surprisingly)
That’s why if we do something open source, it should really play well with whatever other software you want to use. I doubt a project in mid production would switch all of their tasks/tracking to a new/“unproven to them” external closed source software. By all means it needs to have it’s own modules so if I don’t currently have a piece of reviewing software I can use the stock, but if I have codeReviewer or ReviewBoard it should somehow talk to it, or at least have the hooks open for someone that wanted to make them talk.
Hi,
Great initiative! I was thinking of my next home-project and I haven’t decided what to do yet so I would love to join this one. Sounds great. I have worked for some time with python and done some stuff with php, mysql, html and css so i think i can help you guys.
GO GO GO!
[QUOTE=lkruel;8381]I’ve looked at shotgun before, and it’s really impressive, especially if you’re in a film production.
My only issues with it were that it wasn’t too game friendly from what I saw. The example game they had was for a football style game, and I worked on Madden/NCAA and could tell you that’s not how I would set it up =) Maybe the folks that are there now disagree with me.
The other issue was that it seemed like you had to switch to shotgun all the way, and in most studios that’s really hard to do. If your production team is used to using a piece of tools that they’re happy with, it’s really hard to convince them otherwise. But all of the asset management/task tracking/bug reporting were in one place, which is nice but hard to sell (surprisingly)
That’s why if we do something open source, it should really play well with whatever other software you want to use. I doubt a project in mid production would switch all of their tasks/tracking to a new/“unproven to them” external closed source software. By all means it needs to have it’s own modules so if I don’t currently have a piece of reviewing software I can use the stock, but if I have codeReviewer or ReviewBoard it should somehow talk to it, or at least have the hooks open for someone that wanted to make them talk.[/QUOTE]
I agree. I remember shotgun being much more geared toward film/vfx . I don’t think it had file versioning by itself either.
Allright so how do we do this? I’m about to start something similar for work and if there’s any cross pollination work that could happen I’m all for it.
Here’s what I’m thinking:
Webserver running Django(python) and a MySQL database (database can be swapped out in the django settings easily), Django’s ORM is pretty similar to sqlalchemy and actually has given me less grief with lingering connections than sqlalchemy.
The backend is pretty straight forward, just need a few objects like Task, Asset, Component, and a few ancillary ones. The way I’d set it up would be an Asset is a character, and the components would be the source/binary files. Then we can extend those classes out into things like Features, Bugs, Animations, Characters, Props.
The part that gets tricky is managing existing assets (spiderbot to smoke them out) and the linking up of an asset with a task (drop down interface)
It’s honestly just a assload of html and javascript, and a lot of relational database work. Where we keep track of who’s working on what and when do they expect it to be done.
That’s the “project management” part of it. Ideally I’d like to have the project management module (that you could use to sync with an external depot like TFS or Bugzilla), an asset management module (that keeps track of the assets and where they live/screenshots/validation/review). And I’m probably forgetting more stuff, but each module would have submodules that are “features” like reviewing, screenshotting, crawling to find new ones that went unmanaged, validation, creation from templates…
What other features would your studios benefit from? We have enough heavy hitters in this forums that we can build something that hasn’t been seen before. Most studios at the scale of hundreds of employees have outgrown most commercial tools, so it would be awesome to build something for us =)
I’d like to see a way to take designer information into consideration and see how the assets are actually being used in the game. But that starts getting REALLY project specific as a sport title and an MMO would have little in common.
As well as a metrics module that crawls through logs and stores the information in digestible format, but those are really one offish. Things like build logs, player metrics, etc.
Allright, enough stream of consciousness rant, what are you guys thinking of? Am I way on the other side of the spectrum or you guys’ projects can get pretty gnarly to maintain too?
“Most studios at the scale of hundreds of employees have outgrown most commercial tools”
There’s the rub, to be sure. Commercial tools are demo’d (and sold) using tiny samples.
Yeah, let me clear that up. Most commercial “pipeline management tools”. Sure TFS, Perforce and others can handle whatever you throw at them.
And yeah, that’s the tough part since teams that big usually need specific processes to run smoothly. And there’s no “right way”
I don’t know what you guys think is best, for me I would like the app to be a pyqt app that connects to a db, i know nothing about the web and http stuff, and i wanted to be able to hack this stuff out fast.
On the top requirements for me is not the ability to ‘use it from anywhere any any platform’. How do you guys feel?
Anyone with db experience have any feedback about what software is used?