Frog Media Server

#1

Hello all, I have been working on a media site for artists in a creative dev environmental like games or film. Basically its a single place to find and view images and video from your teams. I wrote something similar while at blizzard and they found it irreplaceable and still use it today. If your team is still using a network share for viewing images, you really should consider this or something like it. If you are using something, i’d love to see it, always good to get new ideas. And if you have any feedback, i’d love to hear that too.

Thanks

#2

wow looks nice. I’ve started working on something similar, but that’s much nicer - and it’s written in Python!

I just have some questions though…

  • can I point it to, e.g. one or more SMB network shares and it takes it as source for media?
  • is it easy to set up multiple instances of this on a single web-server? We have some projects where artists not on that project shouldn’t see the media.
  • which formats are supported, and how easy is it to add support for new formats? (we have some HDR/linear PSDs which have been a pain to add to our current web based media manager).

thanks

#3

Thanks! Let me know how yours is coming along.

  1. Right now im using my own methods for organizing the media. Everything needs to be funneled through a url so it goes to the right place. In my experience trying to sync/track a network share is a pain. There are too many ways an artist can break it. Just give them a big red button in maya/photoshop to send their works and they’ll forget there was even a network drive.
    You can easily write a simple batch import script to populate it.
    I have it running at work so that the web server is simply there to serve, the db is on a dedicated machine and the file system is a windows share that’s backed up by IT. So the webserver could catch fire and it would be trivial to get back up and running.

  2. I have this on the books to implement, but adding something like django-guardian would be easy. Then you can give specific user groups permission to view certain galleries.

  3. Right now its just JPG, PNG, GIF, and MP4. Since it’s web based, javascript has to be able to read the images. Im just putting the pixel data onto a canvas, so i guess if you could get the webserver to send the right bytes, it could work with anything. An for video, just stick with h264. i heard somewhere ff is supposed to get support and finally end the format war.

#4

Thanks for the explanation. Sounds useful, but not exactly what I’m looking for.

I’d need something like a multi-user xnview.
Currently we have network shares where we copy texture libraries from clients or bought ones from DVD. As you said, browsing those takes a lot of time, and pointing a local version of xnview to that kills the network if everyone does it, and takes long.

Basically I point the webserver to that folder and tell it: “index it and create thumbs”.
It then makes a tree like menu on the webpage and you can browse all the pictures - there’s a thumb for each and on mouse-over you see the alpha. There’s also a download link for the source file.
It supports PSD, TIFF, TGA and the usual web formats.
Refreshes happen either forced or automatically since there’s not a lot of changes to the libraries.

The interface is a bit rough, there’s no tagging, or favorites, etc. That’s what I liked in your solution. The previewer and the interface seem to be quite advanced and it looks like a very mature product.

#5

With this, adding support for TIFF and TGA would be easy, just create a PNG from the source. This is what i do for PSD, but i have PS do the heavy lifting. The alpha is a cool idea :slight_smile:

As for trees and stuff, after a certain point, menu navigation is useless. Its just a cumbersome as navigating through finder/explorer. Search is the way to go. Tagging helps search. In Frog, i have a system almost ready where you’ll get buckets to add tags/search terms to so you can get AND / OR queries. You’ll be able to drag the tags to different buckets to kind of pivot the search terms. but 90% of the time, folks just type in a search term to find what they want.

I was surprised at how few solutions there are for large image viewers. Everything i found was flash and no one likes flash :slight_smile: So i wrote this one. Obviously its intended for internal use as the bandwidth costs would kill a public site.

#6

Cool stuff! Going to see if the artists have any value for it, if they do I’ll try and get it set up and post here. If not, I’ll keep it in mind and try it out at some point anyway :slight_smile:

#7

Yeah, feel free to hit me up if you have questions if you do end up getting a green light

#8

Is there an intended workflow to keep track of source files? In this solution there is no explicit link between the authored source images and the images on the media server, correct? I’m not criticizing, just making sure I understand :slight_smile:

I think the tool you’ve made looks absolutely delightful as a frontend to finding and categorizing images. It’s made me wondering of the feasibility to hook in a replacement backend that generates thumbnails from traversing the source control repository and automatically adding concept art (and maybe textures too) via checkin-triggers. Such a workflow would delegate the media server to be more of a passive viewer that is always up to date, sort of like autodocumentation for code.

I’ve been Django-curious for a while now, hoping to make some spare time to look into this.

#9

So the workflow for the site is a single URL that accepts

  • a file as the main image, usually a PNG as it’s lossless, but any other web format will work
  • Author/artist
  • local file path, used in search criteria
  • tag list, optional
  • gallery list, optional
  • Other files. these could be source PSDs, TIFFs, DDS, hell anything else you want to link to the image

All this is stored in one folder next to the image and thumbnail, so requesting all this is trivial. I have a task to add to the “Download sources” menu item to just zip up the contents of the folder. right now it just grabs the designated source image.

I’ve tied this URL using python-requests in photoshop and in maya. You could hook this into anything really, as long as you can fire off a proper http request.

#10

Yeah after looking everything over, we’ve decided to give it a try. We’ll probably try it out in the next month or so- our artists were excited. Will see how things go.

#11

awesome! like i said if you need any help getting it up and running, please let me know. I also have a good PS script so artists can hotkey the upload.

#12

Cool project!
I’m testing it here but my lack of knowledge with Django is causing some issues. I’ve managed to get it to connect to a mysql database but I’m not sure how I can populate it?
It complains about “frog.django_content_type” table not existing. Is that something I should create manually or should the code create that for me?

EDIT: Ok managed to sort that out by adding the INSTALLED_APPS into settings.py.

Next problem is I get to the 500.html template after login (had to mess around with the TEMPLATE_DIRS to get it to load any page at all)

I’ll keep digging :slight_smile:

EDIT2: Stuck again. I wonder where it creates all the frog specific tables? Am I missing something or does it not create its own tables?

#13

In Django, you create and sync (to application “models”) the database via administative (console) command:

>python manage.py syncdb

The file manage.py can be found inside the parent folder (Django 1.4 or later) or in the same folder (Django 1.3.x and earlier) that contains the file settings.py.

But make sure first that the database settings in the settings.py file is correct.

#14

Thanks for the reply madyasiwi. I managed to figure out that part so I could sync my db and get the auth tables and a couple of django tables. But my problem is that I don’t get any of the needed frog specific tables.

C:\Development\Python\Django\frog>python manage.py syncdb
Creating tables ...
Installing custom SQL ...
Installing indexes ...
Installed 0 object(s) from 0 fixture(s)

Any ideas on how to solve it?

EDIT: There seem to be some initial data in the fixtures folder, shouldn’t that sync and create a table for me? Even though it’s only one example with the frog.gallery model it should still sync and create that table I would assume.

#15

That has all the steps to setting up a new server. I’ve been trying to write a bootstrap for this, however my bash/powershell scripting knowledge is not my strong side :slight_smile:

Here is a quick and dirty, step-by-step list I wrote up for someone else for a windows setup:

setuptools = http://pypi.python.org/pypi/setuptools
pip = https://raw.github.com/pypa/pip/mast...rib/get-pip.py
just run get-pip.py with python, it’ll do the rest

  1. Get setuptools and then get pip
    This really helps getting stuff installed quickly
  2. Go into %userprofile%\Djangosites. We’ll start a new one for the sake of thoroughness
  3. > pip install django
  4. > pip install django-frog
  5. > pip install south
  6. > pip install PIL
  7. > python django-admin.py startproject dev
  8. Download and install Mysql from http://dev.mysql.com/downloads/mysql/
  9. > mysql -u root -p
  10. Enter the password you used for the mysql setup
  11. mysql> create database frog;
    mysql> create database dev;
    mysql> create user ‘django’@‘localhost’ identified by ‘django’;
    mysql> grant all on dev.* to ‘django’@‘localhost’ identified by ‘django’;
  12. mysql> quit;
  13. > cd dev
  14. > mkdir static
  15. Edit dev/settings.py
  16. Enter this for the DATABASES
    DATABASES = {
    ‘default’: {
    ‘ENGINE’: ‘django.db.backends.mysql’,
    ‘NAME’: ‘frog’,
    ‘USER’: ‘django’,
    ‘PASSWORD’: ‘django’,
    ‘HOST’: ‘’,
    ‘PORT’: ‘’,
    }
    }
  17. Set the following
    MEDIA_ROOT = ‘%userprofile%/Djangosites/dev/static/’
    MEDIA_URL = ‘http://127.0.0.1:8080/static/’
    TEMPLATE_CONTEXT_PROCESSORS = (
    “django.contrib.auth.context_processors.auth”,
    “django.core.context_processors.debug”,
    “django.core.context_processors.i18n”,
    “django.core.context_processors.media”,
    “django.core.context_processors.static”,
    “django.core.context_processors.tz”,
    “frog.context_processors.media”,
    )
    INSTALLED_APPS = (
    ‘django.contrib.auth’,
    ‘django.contrib.contenttypes’,
    ‘django.contrib.sessions’,
    ‘django.contrib.sites’,
    ‘django.contrib.messages’,
    ‘django.contrib.comments’,
    ‘frog’,
    ‘south’,
    )
  18. > mklink /D %userprofile%\Djangosites\dev\static\frog %userprofile%\Djangosites\dev\frog\static
  19. > python manage.py syncdb
  20. > python manage.py schemamigration frog --init
  21. > python manage.py runserver
#16

We got an initial test version set up here and the Django/setup portion was the most difficult. But it’s totally worth it once it’s working! Hopefully as more people use it we can improve the documentation and setup process.

#17

I agree, i need to make a bootstrap for a super quick windows server to demo to the brass and then one for centos/rhel and ubuntu.

Glad you guys like it :slight_smile:

#18

I’ll gladly share my notes on the setup process when I can find some time to actually finish the setup. I’ve manage to get almost everything working except actually showing any pictures :stuck_out_tongue:

I’ll bother you again over email sometime this week. I was missing libjpeg and I’ve reinstalled pil with support now, should I reinstall anything else to get it running?

#19

nope, just pixel shift your original image and reupload. It won’t do anything if the image is exactly the same.

#20

Even if I upload a new (never used) picture it doesn’t get added.
It will copy the image to a subfolder under /static/ on the server.
It adds it to the frog_image table.

But it won’t write anything to the frog_gallery_images table, shouldn’t it create a connection there?
If I manually enter a row where I link a image id to a gallery id, the Navigation tab will update it’s count to correctly show one image in the gallery. But it won’t show it, gets stuck on “fetching images…”

Any ideas what I might be missing?

Some output from the server, might give you a hint?


[25/Oct/2012 11:27:33] "GET /frog/gallery?_dc=1351157256557&page=1&start=0&limit=25 HTTP/1.1" 200 962
[25/Oct/2012 11:27:33] "GET /frog/gallery/2/filter?h8po57bn&filters=%5B%5B%5D%5D&models= HTTP/1.1" 500 9449
[25/Oct/2012 11:27:35] "GET /frog/gallery/2 HTTP/1.1" 200 3021
[25/Oct/2012 11:27:35] "GET /frog/pref/?h8po58z0 HTTP/1.1" 200 455
[25/Oct/2012 11:27:35] "GET /frog/tag/?json=true HTTP/1.1" 200 243
[25/Oct/2012 11:27:35] "GET /frog/gallery?_dc=1351157258668&page=1&start=0&limit=25 HTTP/1.1" 200 962
[25/Oct/2012 11:27:35] "GET /frog/gallery/2/filter?h8po58z3&filters=%5B%5B%5D%5D&models= HTTP/1.1" 500 9449