Converting color tables

VisIt can import color tables in a simple XML-like format called a .ct file that you can copy to your ~/.visit directory. When you copy a .ct file to that directory, VisIt will find it at startup and make it available for use.

VisIt's color table format is one which uses parametric color control points and linear interpolation, which it then samples to a fixed color table when necessary. Many other color table formats use these sampled styles, however. Though one could naively convert them to a control point format -- e.g. create 256 control points for 256 sampled colors -- most color tables use smooth gradations of color transitions and could be effectively recreated using many fewer control points, which also makes them easier for users to edit. The script below handles this more optimal conversion process.

ctconvert.py

This is a Python program which takes 3 arguments:

  • number of control points to create
  • input file name in .col, .am, or .icol format
  • output file name, typically ending in .ct and in your ~/.visit directory

For example:

python ctconvert.py 5 glow.col ~/.visit/glow.ct

This will create a new VisIt color table, with 5 control points placed optimally to minimize the error in reconstructing the input sampled color table. When you start VisIt, it will load this color table as "glow".

While it's technically up to you to choose the "right" number of colors, because it's essentially making the optimal placement of those colors, there's little harm in specifying this number a little. So you could just choose something like 10 points where you're probably quite safe on most reasonable color tables. But it also prints the error as it goes -- if you select enough control points to get the error just down below 1.0, that's probably pretty close to a minimal set on a color table of size 256, though again, a couple extra points won't hurt (and this error is hardly a robust metric for a number of reasons).

This could probably be extended quite easily to other sampled color table formats, by the way.

#! /usr/bin/env python

# Program: ctconvert.py
# Creator: Jeremy Meredith
# Date:    February 19, 2009
#
# Convert sampled color tables from one of a few input formats into
# VisIt's format, choosing an optimal selection of some number of
# control points.  (The number of control points is chosen by the
# user, though something between 5 and 10 does well for many
# common types of color table creations.)
#
# It currently supports already sampled color tables in Amira/Avizo
# formats.  It could easily support other sampled color table types,
# noting that if the input isn't already sampled to some number of
# values (like 256), but lives in color-control-point space, there
# are probably simpler and more efficient conversions.  Plus, this
# one really assumes it's a sampled set of colors.
#
# Note: it will output alpha channels, and VisIt can read them,
# but as of this writing VisIt does not use the alpha channel.

import sys, os, math

# getControlPointsForIndex
# - returns the pair of indices of closest control points for given index
def getControlPointsForIndex(ctrl, index):
    pt0 = index
    pt1 = index
    if pt0 == 0:
        pt1 = index+1
    else:
        pt0 = index-1
    while not ctrl[pt0]:
        pt0 -=1
    while not ctrl[pt1]:
        pt1 += 1
    return (pt0,pt1)

# calcError
# - calculates the error between the target
#   color table and the current control pts
def calcError(rgba, ctrl):
    (pt0,pt1) = (-1,-1)
    error = 0
    ptdiff = (pt1-pt0)
    for i in range(len(rgba)):
        if i >= pt1:
            (pt0, pt1) = getControlPointsForIndex(ctrl, i)
            #print "new ctrl pts=",pt0,pt1
            ptdiff = pt1-pt0
        alpha = (i-pt0) / float(ptdiff)
        r0 = rgba[pt0][0]
        r1 = rgba[pt1][0]
        g0 = rgba[pt0][1]
        g1 = rgba[pt1][1]
        b0 = rgba[pt0][2]
        b1 = rgba[pt1][2]
        a0 = rgba[pt0][3]
        a1 = rgba[pt1][3]
        rv =  alpha * (r1-r0) + r0
        gv =  alpha * (g1-g0) + g0
        bv =  alpha * (b1-b0) + b0
        av =  alpha * (a1-a0) + a0
        #print "rv[",i,"]=",rv," orig=",rgba[i][0]
        re = abs(rv - rgba[i][0])
        ge = abs(gv - rgba[i][1])
        be = abs(bv - rgba[i][2])
        ae = abs(av - rgba[i][3])
        re = re * re
        ge = ge * ge
        be = be * be
        ae = ae * ae
        error += (re+ge+be+ae)
    return math.sqrt(error)



# findBestAddDelPoint
# - finds the best point to add (add==True) or delete (add==False)
#   (i.e the one by deleting which increases the error the least, or
#    the one by adding which decreases the error the most)
def findBestAddDelPoint(ctrl,rgba,add):
    lowerr = 9999999
    lowpos = -1
    for i in range(1,len(rgba)-1):
        if (add and ctrl[i]==False) or (not add and ctrl[i]==True):
            ctrl[i] = add
            myerr = calcError(rgba,ctrl)
            #print "   newerr for pt",i," =",myerr
            if myerr < lowerr:
                lowerr = myerr
                lowpos = i
            ctrl[i] = not add
    return (lowpos, lowerr)

# findBestAddPoint
# - finds the location and resulting error of the control point which after
#   adding to the set, reduces the error the most
def findBestAddPoint(ctrl,rgba):
    return findBestAddDelPoint(ctrl,rgba,True)

# findBestDelPoint
# - finds the location and resulting error of the control point which after
#   deleting from the set, increases the error the least
def findBestDelPoint(ctrl,rgba):
    return findBestAddDelPoint(ctrl,rgba,False)

# getNthDelPoint
# - finds the location and resulting error of deleting the nth control point
def getNthDelPoint(ctrl,rgba,index):
    n = 0
    for i in range(1,len(rgba)-1):
        if ctrl[i]==True:
            if n == index:
                ctrl[i] = False
                err = calcError(rgba,ctrl)
                ctrl[i] = True
                return (i, err)
            n += 1
    return (-1,-1)

def read_gb_file(fname):
    """
    Parses a yorick color table(.gb file) & returns expected rgba list.
    """
    res = []
    lines =[l.strip() for l in  open(fname).readlines() if l.strip() != ""]
    start = False
    for l in lines:
        if not start:
            if l.startswith("#  r   g   b"):
                start = True
        else:
            vals = [float(v) / 255.0 for v in l.split()]
            vals.append(1.0)
            res.append(vals)
    return res

# printUsageAndExit
# - print usage and exit
def printUsageAndExit():
    print "Usage:",sys.argv[0],"<numpts> <input> <output>"
    print ""
    print "  numpts >= 2"
    print "  input in .col/.am or .icol format"
    print "  output in VisIt .ct format, typically in ~/.visit/"
    sys.exit(1)

#
# main
#

# argument parsing, file opening
if len(sys.argv) != 4:
    printUsageAndExit()

npts = int(sys.argv[1])
if npts < 2 or npts > 1024:
    printUsageAndExit()

try:
    filein = open(sys.argv[2],'r')
except:
    print "Couldn't open input file",sys.argv[2],"for reading."
    printUsageAndExit()

try:
    fileout = open(sys.argv[3],'w')
except:
    print "Couldn't open output file",sys.argv[3],"for writing."
    printUsageAndExit()

# parse the input file
rgba = []
ncolors = 0
filetype = 0
for line in filein:
    # filetype values:	 
    #   0 means still determining type	 
    #   1 means single-column with alpha and header (typically *.am/*.col)	 
    #   2 means 4-column, with indexes but no alpha (typically *.icol)
    #   3 means .gb file from yorick
    if filetype == 0:
        if line.startswith("# AmiraMesh 3D ASCII 2.0"):
            print "Detected .am style color table."
            filetype = 1
            reading = False
        elif len(line.split()) == 16:
            print "Detected .icol style color table."
            filetype = 2
            rgba = [[] for i in range(4096)] # don't know how long yet
        elif line.startswith("# Gist"): # yorick '.gb' file
            filetype = 3
            rgba = read_gb_file(sys.argv[2])
            ncolors = len(rgba)
        else:
            print "Can't determine file type."
            sys.exit()
    # here's where the actual parsing happens
    if filetype == 2:
        words = line.split()
        for i in range(0,len(words),4):
            ind = int(words[i])
            if ind+1 > ncolors:
                ncolors = ind+1
            rgba[ind].append(float(words[i+1])/255.)
            rgba[ind].append(float(words[i+2])/255.)
            rgba[ind].append(float(words[i+3])/255.)
            rgba[ind].append(1.0)
    elif filetype == 1:
        if not reading:
            if line.startswith("@1"):
                reading = True
        else:
            words = line.split()
            if len(words) == 4:
                rgba.append([float(words[0]),
                             float(words[1]),
                             float(words[2]),
                             float(words[3])])
            else:
                ncolors = len(rgba)
                break

# some file types we didn't know the length; shorten
# the array to the actual read length
print "Length of input data is",ncolors
rgba = rgba[0:ncolors]

# set points at 0 and n-1 (these control points are immutable)
print "Setting control points at endpoints ( indices 0 and",len(rgba)-1,")"
ctrl = range(len(rgba))
for i in range(len(rgba)):
    ctrl[i] = 0
ctrl[0] = 1
ctrl[len(rgba)-1] = 1

# calculate the initial error
lastError = calcError(rgba, ctrl)
print "Starting error=",lastError

# just a warning in case they misunderstood the meaning of npts
# this works, but it's hardly useful in most cases
if npts==2:
    print "Warning: not adding any control points; using only fixed endpoints"

# For each of the remaining npts-2 points, first add a point, then
# shuffle the existing points as much as necessary.  I can't easily
# prove it, but I think this should be effectively optimal.  (Note
# that a faster version, where you simply find the best point to delete
# and then add a new best point, and seeing if those were the same, is
# definitely not optimal in all cases.  I tried it.  This does better.)
for p in range(npts-2):
    (pos,err) = findBestAddPoint(ctrl,rgba)
    ctrl[pos] = True
    print "Setting control point at position",pos,"newerror=",err
    done = False
    while not done:
        done = True
        for j in range(p):
            (delpos,delerr) = getNthDelPoint(ctrl,rgba,j)
            ctrl[delpos] = False
            (addpos,adderr) = findBestAddPoint(ctrl,rgba)
            ctrl[addpos] = True
            if delpos != addpos:
                print "    Moved control point from",delpos,"to",addpos,"reduced error to",adderr
                done = False
                break

print "Final error=",calcError(rgba, ctrl)
print "Writing output file"

# Okay, write the output, and close the files!
print >>fileout, '<?xml version="1.0"?>'
print >>fileout,'<Object name="ColorTable">'
print >>fileout,'    <Field name="Version" type="string">1.11.0</Field>'
print >>fileout,'    <Object name="ColorControlPointList">'
for i in range(len(rgba)):
    if not ctrl[i]:
        continue
    r = int(rgba[i][0] * 255)
    g = int(rgba[i][1] * 255)
    b = int(rgba[i][2] * 255)
    a = int(rgba[i][3] * 255)
    pos = ((i) / (len(rgba) - 1.))
    print >>fileout,'        <Object name="ColorControlPoint">'
    print >>fileout,'            <Field name="colors" type="unsignedCharArray" length="4">',r,g,b,a,'</Field>'
    print >>fileout,'            <Field name="position" type="float">',pos,'</Field>'
    print >>fileout,'        </Object>'

print >>fileout,'    </Object>'
print >>fileout,'</Object>'

filein.close()
fileout.close()

Example color table

Here is an example color table in the .ct format:

<?xml version="1.0"?>
<Object name="ColorTable">
    <Field name="Version" type="string">1.11.0</Field>
    <Object name="ColorControlPointList">
        <Object name="ColorControlPoint">
            <Field name="colors" type="unsignedCharArray" length="4"> 0 0 255 255 </Field>
            <Field name="position" type="float"> 0.0 </Field>
        </Object>
        <Object name="ColorControlPoint">
            <Field name="colors" type="unsignedCharArray" length="4"> 0 102 255 255 </Field>
            <Field name="position" type="float"> 0.0909090909091 </Field>
        </Object>
        <Object name="ColorControlPoint">
            <Field name="colors" type="unsignedCharArray" length="4"> 0 204 255 255 </Field>
            <Field name="position" type="float"> 0.181818181818 </Field>
        </Object>
        <Object name="ColorControlPoint">
            <Field name="colors" type="unsignedCharArray" length="4"> 0 255 204 255 </Field>
            <Field name="position" type="float"> 0.272727272727 </Field>
        </Object>
        <Object name="ColorControlPoint">
            <Field name="colors" type="unsignedCharArray" length="4"> 0 255 102 255 </Field>
            <Field name="position" type="float"> 0.363636363636 </Field>
        </Object>
        <Object name="ColorControlPoint">
            <Field name="colors" type="unsignedCharArray" length="4"> 0 255 0 255 </Field>
            <Field name="position" type="float"> 0.454545454545 </Field>
        </Object>
        <Object name="ColorControlPoint">
            <Field name="colors" type="unsignedCharArray" length="4"> 102 255 0 255 </Field>
            <Field name="position" type="float"> 0.545454545455 </Field>
        </Object>
        <Object name="ColorControlPoint">
            <Field name="colors" type="unsignedCharArray" length="4"> 204 255 0 255 </Field>
            <Field name="position" type="float"> 0.636363636364 </Field>
        </Object>
        <Object name="ColorControlPoint">
            <Field name="colors" type="unsignedCharArray" length="4"> 255 234 0 255 </Field>
            <Field name="position" type="float"> 0.727272727273 </Field>
        </Object>
        <Object name="ColorControlPoint">
            <Field name="colors" type="unsignedCharArray" length="4"> 255 204 0 255 </Field>
            <Field name="position" type="float"> 0.818181818182 </Field>
        </Object>
        <Object name="ColorControlPoint">
            <Field name="colors" type="unsignedCharArray" length="4"> 255 102 0 255 </Field>
            <Field name="position" type="float"> 0.909090909091 </Field>
        </Object>
        <Object name="ColorControlPoint">
            <Field name="colors" type="unsignedCharArray" length="4"> 255 0 0 255 </Field>
            <Field name="position" type="float"> 1.0 </Field>
        </Object>
    </Object>
</Object>