Creating a Pipeline?

[h6x6n]

This may be a little late for this, but my two cents:

the way you’re organizing your textures is a bit unnecessarily restrictive, at least as long as you don’t have additional metadata available for the images. I’m much more a fan of nonexculsive types of information (think tags in wordpress vs. categories). You could potentially use both systems (your current directory-based approach for the “material” (concrete, wood, dirt, etc.) and then tags for that and additional data), but that seems a little unnecessary.

you could set this up using metadata, either in the image file itself (TGA supports this…i think, but I don’t know what’s available in python for doing that), in a database, or in a sidecar file that holds metadata about the file.

and on the issue of documentation generation, check out sphinx. The python docs themselves are actually generated using sphinx.

[SFXRogue]

Thanks Tyler, I started looking into some modules that will allow me to add meta data to images. PIL has some of this functionality but it seems there might be better, more detailed modules out there.

Once I find one I like I will probably switch the texture library over to meta data instead of separate directories. After I get the metadata working they way I would like it to I’ll be able to rewrite the part of my texture browser that sorts the images and then start on the texture submit tool. This tool will allow users to submit new textures to the library and give them the ability to add custom tags to the photos.

Sphinx seems awesome as well and am definitely looking into that!

This week most of my efforts were focused on documentation generation, setting up the wiki we will be using and documenting some of my own processes (Videos/written content) on the creation of this all.

The wiki we are using is currently set up and I’ve started to migrate some of the documentation from the older site to this one. This new wiki will be used as a container for all tutorials and tool documentation/revisions. Currently it isn’t open to any users as I migrate and set up some of the information but once the pipeline is in place we will have some moderators to review the content being added.

Aside from the wiki/documentation/process most scripting time this week has been used to clean up some of the existing scripts, make some changes, fix bugs and overall just make things look nicer. (Organization!:D:) I’ve been gathering a lot of feedback as well about all of it have have begun consolidating the feedback into main categories which I will use to review for myself and then make changes.

Since this upcoming week will be GDC I wont be able to post an update but will hopefully be able to talk to some of you in Person! The week after though I will hopefully have a lot more to share and will also post some of my process documentation.

[SFXRogue]

That was a long 2 weeks but finally found some downtime to post this update! After GDC last week I got a ton of new ideas for new tools that we could possibly use and way to improve the pipeline even further. First I’d like to say Boot camp was AWESOME yet again this year and definitely full of very useful information that I was able to take and apply to what I’m currently doing.
Here’s a quick consolidated list of the changes I plan on making or tools I plan on building before I get into it:

Tool Search System
Project Viewer for Maya
Utility Script Manager
Meta-Data Tags for libraries
Perforce utility in Maya (Submit/Check out)
Documentation

Tool Search System:

I’d like to implement a system within Perforce and Maya with the ability quickly search through all the tools we will have on the server. This might be done through simple tagging of the scripts or just using their already sorted directories on the server. I know that within Maya I am going to be sorting the tools in the drop down menu so that it isn’t just one long list of random stuff. Currently half of the tools that are already on the server are organized this way so it’s just a matter of cleaning up the grouping and adding the other tools that have not been sorted yet into the appropriate categories.

Along with this I was able to get some feedback at the roundtables as to how tools are distributed to artists. From this feedback I decided that we will have 2 main script directories. The first has all scripts that are being worked on and have the possibility to be changed, while the second will have stable versions of the scripts. This directory will be updated each morning where I will just have an automatic process copy scripts over from the first directory to the second. The tool shelf in Maya that I will set up will also only draw from the more stable directory allowing us to have a bit of version control, and know who has what tools.

Project Viewer for Maya:

This tool is based off of Jeremy Ernst’s project viewer her created for his pipeline work. I figured this could be useful to our instructors for days when we would have critique on our current models or work. Instead of having to go through perforce to check each file out this lets them browse to a specific project in a class and from there just choose which file to view. Simply just cutting out some time from the task and making it a bit easier to interact with the server. This was also something I had fun putting together the night I got back from San Francisco! The image below shows the layout, it works similar to the project creation tool in the sense that they can choose by year (soon), class, section, and then project. Some of the functionality still needs to be worked out.

Utility Script Manager:

The Utility Script Manager would be a simple tool that allows the admin to execute utility scripts easily, such as group cleanup, project archiving, project deletion, ect. This would simply just give the user a dropdown menu of all the utility scripts in the admin’s folder, on choosing one the user can then run the script and then view the resulting file (if there is one). Essentially a simple browser for the collection of scripts I am starting to amass on the depot. Below is a simple diagram of how I’d like it to work.

Meta-Data Tags for Libraries:

As this says I am going to be switching our current texture library over to Meta data (JSON) so that tagging and tag searching can be used. This week I have been doing some research on JSON and am deciding on the types of tags I would like to use with our image library. With the use of Meta data now I have also changed the structure of that depot yet again (third time’s a charm!). The texture library is now structured as such:

//Assets/Libraries/Textures
//Assets/Libraries/Textures/Images
//Assets/Libraries/Textures/Thumbnails

In the Images directory is where all of the Meta data and actual size images will be kept, where the Thumbnails directory will have thumbnails of all the full size images. The reason for having thumbnails of all the images is that they will be used for the visual texture browser. This way instead of generating thumbnails whenever the texture browser is run it just simple draws from the cache of images already on the server.
Along with the tagging system I have also started to lay out an application for users to submit new textures to the library. This system will allow them to tag their images with existing or new tags before they are added in with the others. I also plan to have it check to see if the texture they are submitting is identical to any images currently in the library, preventing duplicates. Next week I will hopefully have a simple layout of how I’d like this to look and function and will be able to share it for some feedback.

Perforce Utility in Maya:

Something simple but on the to-do list of things that need to be done. I plan to echo how Jeremy Ernst did his Perforce functionality in Maya since I feel it was simply and elegantly done. This just consists of 3 buttons, Check-out, submit, and sync. All self explanatory but will give the users of the pipeline an easy way to check their files out from the server, check to see if it’s the latest version and if not have it sync up.

Documentation:

Finally documentation yet again and after GDC we’ve decided to switch over from just using a Wiki to using Sphinx and Wiki together. Both currently being looked into for how to generate content and manage users (specifically for the wiki). As I’ve said in an earlier post as well the Wiki will be used to house tutorials, tool documentation, versioning, and history as well as breakdown of actual code hopefully through Sphinx.

Well that was a good mouthful! As always I’ll be continuing to create and make progress on all of the tools and pipeline overall. Right now I feel like I have a pretty good amount ahead of me for the next few weeks and I’ll also have some good updates to follow in suit with those.

As a side note project I’ve also begun to re-write my Simple Weapon Generator or SWaG as I like to call it for UDK. This tool allows for quick creation of simple weapon scripts to be compiled for UDK. The user simply just picks a weapon template, sets the weapons name and then fills in the fields on the form. Things such as weapon damage, firing sound, fire rate, ect. I can post a separate update on this if anyone is interested. Coupled with this I also have a Simple Weapon Rig setup to accompany the script. The rig is just placed within the gun with the default naming scheme of joints used in UDK for the hands and muzzle flashes.

Thanks again for those that read and the support I received at GDC. It was definitely a refreshing and re-energizing experience that was much needed!

[SFXRogue]

Somehow I managed to let an entire week get away from me without posting an update, time to fix that! Things have been a bit hectic but I’ve been able to cross a few things off the list I had last week which is always nice!

To start, the texture browser/submit is starting to come together nicely with a UI and some basic functionality. A fair amount of time has been spent trying to figure out a nice layout so that it will be simple and easy to use for everyone. What I came up with is below. I’m still a bit unsure about the texture submit layout but it can always be tweaked. (Please ignore the frame4 title, I haven’t cleaned up all my code yet.)

The texture browser is fairly straight forward with how it works. Look for textures, receive bacon. The submit is where I still think it needs a bit of tweaking in the layout but for now as a working prototype it’ll do. Users select a texture type and can add some tags. Then when its submitted it will generate a meta data file based on the tags added. A lot of the functions right now are also just placeholders (very simple) but I will be refining them this week, especially the tag searching system for parsing through all the JSON files.

Along with getting some pipeline work done these past 2 weeks I was also able to fix up an old tool of mine, the Simple Weapon Generator. This was a set of 2 tools, 1 rigging and 1 for making scripts. Its use it to allow for the creation of simple weapons in UDK without having to know unreal script. Below are some screenshots of the tools.

The SWAG Rig creates the bone structured used for weapons in UDK. The user just places the locators at the specific locations on their mesh and then clicks Create Rig. This destroys the locators and creates a skeleton to bind the mesh to. It then just needs to be imported to UDK and have the sockets added.

Simple Weapon Generator creates the scripts needed to have the gun work in game. All that needs to be done for this is to fill out the appropriate information on the forms. Ive even included a simple FAQ under file to explain the values and how they are referenced in engine. Upon clicking the create scripts button it will create the 5 needed unreal scripts for the weapon. These then just need to be placed in the “mymod” folder and compiled to work.

Ultimately I’m hoping to expand onto this with more weapon options and possibly a variation for simple vehicle creation and rigging. Not directly related to the pipeline but definitely something extremely fun and useful for most of my peers.

As time dwindles down on the semester there are only a few more “key” elements to build and incorporate into this pipeline before time runs out. Mainly right now its going to be getting the texture library working optimally, getting Maya perforce integration working nicely, making a few tutorials, and creating a more official documentation of the scripts.

It seems to be all about the grind and putting in the man hours to produce whats needed at this point in these final 5 weeks. Hopefully I’ll be able to have a base pass on everything by next week so the rest of the time can be spend making everything I have more efficient and ready with documentation. So much to do, so little time Gets more coffee

[SFXRogue]

The end of the semester brings so much chaos, but I’m glad I can finally do an update. (Wheres that magic no sleep pill!) To list things out I was able to get a stable version of the Simple Weapon Generator out to the public and to my peers, finished programming on a senior thesis project that needed the help, and also had some good progress on the pipeline that included almost finishing the texture browser and script depots. So close! Admittedly though, much of my time this week was messing around with unreal script and getting some good interaction with artists and iteration of tools which is invaluable to me as a training TA. I’ll start off with the progression of Simple Weapon Generator.

Thanks to a teacher and his class I was able to rapidly prototype and test the Simple Weapon Generator I made. The feedback I received from him and the students really helped move that tool forward to something that was more useable and had much more control. This was a great experience because it’s what I can assume is a good interaction between TA and artists will be like in the industry. Starting with the most basic form of SWAG, they were able to instantly give me feedback on the layout of the tool and features they would like to see in it. This back and forth happened 3-4 times before all things were sorted out and it seemed to be a good, stable running application that they could use and it would fulfill their needs. Apart from a few extra items they wanted this turned out to be a good tool that I will probably work on in the future.

Currently it is posted to the forums and I plan to do a quick run-through/demo video to accompany it soon. ( http://forums.epicgames.com/threads/902158-Simple-Weapon-Generator-Tool )

This process I can see being directly related to the industry and workplace as being a normal interaction for a TA. Things went smoothly and I quite enjoyed it as opposed to my next encounter.

The second big thing I was a part of this week was helping a senior thesis with making working, flyable vehicles in UDK for them. I first took this task because they asked nicely and it gave me a pretty good excuse to just dive right into Unreal Script since I’ve had an urge to learn it for a long time and jumping right in is the perfect way to do this! I immediately started working on the problem purely by just poking around at the Unreal Script and unraveling it from there. This started with one of the main .uc files of a flying vehicle and from there I traced the classes back and figured out the main 5 or 6 scripts I would need to make a working vehicle in UDK. After about 2 nights of straight work I began to understand how to get the vehicles to behave properly but more importantly I was learning Unreal Script in the best way possible, and that’s by purely just tinkering around with code and figuring out what works and what doesn’t. This seems to be the best tactic for me.

The big thing about this project though was not really the work but being able to work with a senior team, which at first was great but then went a bit south. As my teacher mentioned to me me, what we have created seems to be a monster in and of itself. It was never there to begin with and no one really knew they wanted it, but as soon as they saw what it could do they wanted more. Possibly more than I could have offered the team due to my own assignments in the major. Essentially what happened was things would go wrong with their vehicles (minor bugs) and they would automatically assume that since they had this thing that it would always be working and be bug free. It also happened that whenever something would go wrong they would automatically pin me for the problem, which in most cases was not true, it was something on their end that was broken. The worst part being they would not go through the proper steps of debugging in order to figure out the exact problem. I was usually given short, vague responses or feedback of the problem and I would have to check the entire system to be safe. While this probably was a good thing to experience (seeing the other side of the coin), its something I hope to avoid in the future. /rant

So in summary I was able to experience both sides, the good and bad of being a tech artist. But its also great to see that people want to take advantage of what I’ve been creating, be it good or bad experiences. They’ve gotten a taste of something they never knew they’d want to try and now they want to experience more of it and see bigger and better things in the pipeline.

Onto the more objective side of things the Texture Browser is operation at a basic level! We can now browse by thumbnails and basic search types. To get this functionality though I did have to rewrite part of an existing module. This was a great practice because I had to go through and read someones code then find a way to write in the specific functionality without breaking anything. Luckily it all worked out and I was able to add an extra flag to the ShowDir function which allowed for list inputs of images instead of anything within a certain directory. This helps with being able to mask off images during the search. The only stopping point right now is switching JSON to a database and reading off of that. If we kept the calls as they were right now it would mean a possible for loop being nested 2-4 deep, and thats just no good on processing especially with images involved. I hope to fix this problem soon but it’s definitely stepping into a new arena for me. The database will be done with sqlite3 and Django.

These last weeks will be used to finish up the Texture Browser (almost there!) and writing some documentation, which I hope to knock out soon. That list includes:

-Vehicle Documentation in UDK
-Weapon Documentation and tutorial in UDK
-Simple Weapon Generator tutorial
-Documentation of all the tools for the pipeline
-More process

Crunch mode activated!