CodeBetter.Com
CodeBetter.Com
RSS 2.0 via Feedburner
           Do you Twitter? Follow us @CodeBetter

Steve Hebert's Development Blog

Steve's Blog - From .Net to dotMath and everything in between.

November 2004 - Posts

  • Joel Spolsky Conversation on Thursday

    I just got an invite to Joel Spolsky's talk on usability and social interface design.  I'm looking forward to this one.
  • off-topic: nastalga alert - Commodore 64 in a joystick game

    I read about a company building a commodore 64 in a joystick with ~30 games back in June.  It turns out - according to slashdot - that QVC has bought up some 250,000 of these units to exclusively sell them.  I think I just added an item to my Christmas wish list.

  • If you won the lottery, what kind of programming would you do?

    I was reading Steve Eichert's blog entry asking “if you had it to do all over again would you go into computers?“  Then I started thinking, if you won the lottery what kind of programming would you do?

    I'd probably make an offer to John Deere to do work like this for free out of my house. They take real-time yield information from a combine and drive the equipment accordingly.  Then they can take this information the following year and feed it into the planting equipment to vary seed density and fertizilizer to maximize yield over the field according to soil quality (and other factors).  That is so freakin' cool!

    Added thought:  Add a little GPS drive system and farmers could be using this interface for their field work.

    Otherwise I'd do open-source software and miscellaneous projects for businesses that I find compelling and try to turn them into good business opportunities in my spare time.

  • Effective OLAP/Analysis Services Books

    I've picked up a couple books on Analysis Services including the 'MDX Solutions' book by George Spofford.  While I'm happy with these books, I'm really looking for something different, but I'm not sure how to quantify it.

    Back in '92, I wrote an “Analysis Services” style application that used dimension tables, a fact table, a cube abstraction and even query caching.  Of course, back then the terms like “fact table“ and “dimension table“ didn't exist but the functionality was identical.  The creation of these systems is methodical and interesting when you consider the types of data situations you can run into.  Bottom line, it's a relational database problem and I've always felt that OLAP was a clever way of saying you're “denormalizing“ the database without digging up a religious war with the “n-th normal form zealots.“

    The problem with todays set of books is that authors tend to get caught up in a methodical rundown of OLAP theory in presenting their how-to approach.  OLAP theory is comprehensive and necessary - it's there for a reason.  But I've got to think there is a book that cuts through the theory and develops a core understanding of using the tools to build a solution from a conventional dbms perspective and THEN role into the theory bits.  For example don't ramble on talking about Star Schemas - show the reader an implementation, why it works, why it's necessary and then reveal that the user just learned the Star Schema bit. Expose some details to tie the lesson together and move on.

      We used to refer to great books as being “content-dense”, where you can sit down and read a chapter and feel like the author has revealed great truths about a language/os/design/etc..  I just don't get that “I'm getting smarter by the minute feeling” with these books.  The OLAP books I've seen tend to be content-sparse - one or two facts surrounded by a chapter of discussion. Yawn.

    That said, I am happy with Spoffords book.  His book focuses on MDX which is a very related but different problem set.  I'll continue looking for a more general book and blog it if I find it...

  • MS Whitepaper on CLR Integration with SQL 2k5

    Microsoft now has the “Using CLR Integration in SQL Server 2005” whitepaper on their site.  This is a good read and discusses the operational differences between T-SQL and managed code in the SQL Server environment
  • Integrating database upgrades into project distribution

    My next step in refining our build process is incorporating the database upgrade script into our msi distribution.  I've got the code that selects 1..n servers/databases and performs the upgrade. Next, I need to create a build function that encrypts and compiles the script into a resource file for the custom installer script to reference.  I like the fact that the encryption library performs some compression for the sake of security, but I'm curious how the rates compare to commercial zip packages.

  • Building a database script using NAnt - supporting source code

    I had a couple of requests for the code I used to build my sql script on the fly in conjunction with NAntContrib (blogged about it here).  Before I show the code, my next refactoring would include (1) creating a ProcessState interface and generating seperate ProcessState classes for NAnt and MultiThreaded environments and (2) do the same for vssManager to handle multiple source control engines.

    Before I list the code, I suppose I should add a CMA disclaimer:

    The code listed below is offered without any warranty expressed or implied.  This is despite anything stated before, after or during this posting.  To understand the scope of this lack of a warranty, understand that no guarantee is made that code will compile and if it does compile and actually accomplishes anything it is merely a coincidence on par with a monkey randomly banging on a typewriter and producing a word-for-word copy of Hamlet.  If you use the code and it causes you any problems, you assume full responsibility for the problems and obsolve the author of any form of responsibility.

    I should have been a lawyer... now for the code...

     

     

    using System;
    using System.Collections;
    using System.IO;
    using SourceSafeTypeLib;
    using System.Xml;



    namespace hssScriptBuilder
    {
    /// <summary>
    /// Summary description for hssSourceControl.
    /// </summary>
    public class hssSourceControl
    {
         public class ProcessState
         {
             /// <summary>
             /// Stateful information used for current status displays and caching
             /// of progress data between worker and ui threads. This data must be
             /// protected for multi-threaded access.
             /// </summary>

             private ArrayList m_alProcessNotes = new ArrayList();
            
             private String m_sCurrentSection;
             private String m_sCurrentItem;

             private int m_iSectionCount = 0;
             private int m_iCurrentSection = 0;
             private int m_iItemCount = 0;

             private bool m_bStop = false;

             /// <summary>
             /// Project Property: set/get of NantProject for displaying run
             /// status over the Nant channel.
             /// </summary>
             private NAnt.Core.Project _nantProject = null;

             public NAnt.Core.Project NAntProject
             {
                get { return _nantProject; }
                set { _nantProject = value; }
             }
            

             /// <summary>
             /// Stop: Allows for notifying the worker thread to quit processing
             /// the Source Control files.
             /// </summary>
             /// <param name="bStop"></param>
             public void Stop( bool bStop )
             {
                m_bStop = bStop;

             }

             /// <summary>
             /// IsStopped: Used to determine if the worker thread has been requested
             /// to stop processing.
             /// </summary>
             /// <returns></returns>
             public bool IsStopped()
             {
                return m_bStop;
             }

             /// <summary>
             /// GetItemCount(): returns the number of processed files during the course
             /// of a run.
             /// </summary>
             /// <returns></returns>
             public int GetItemCount()
             {
                lock(this)
                 return m_iItemCount;
             }

             /// <summary>
             /// GetCurrentSectionIndex(): returns the current section number being process.
             /// A section is defined as a single line item in the SCC processing XML document.
             /// </summary>
             /// <returns></returns>
             public int GetCurrentSectionIndex()
             {
                lock(this)
                 return m_iCurrentSection;
             }

             /// <summary>
             /// GetSectionCount(): Returns the total number of sections declared
             /// for a given run.
             /// </summary>
             /// <returns></returns>
             public int GetSectionCount()
             {
                lock(this)
                 return m_iSectionCount;
             }

             /// <summary>
             /// GetCurrentSection(): Returns the name of the current section being processed.
             /// </summary>
             /// <returns>String</returns>
             public String GetCurrentSection()
             {
                lock(this)
                 return m_sCurrentSection ;
             }

             /// <summary>
             /// GetProcessNotes(): Returns an ArrayList of messages created by the
             /// worker thread since the last time GetProcessNotes was called.
             /// </summary>
             /// <returns>ArrayList of strings</returns>
             public ArrayList GetProcessNotes()
             {
                ArrayList aList = new ArrayList();
                lock(this)
                {
                 IEnumerator oEnum = m_alProcessNotes.GetEnumerator();
                        
                 while( oEnum.MoveNext() )
                     aList.Add( oEnum.Current );
                        
                 m_alProcessNotes.Clear();
                }

                return aList;
             }

             /// <summary>
             /// GetCurrentItem(): Returns the name of the current item being processed.
             /// </summary>
             /// <returns></returns>
             public String GetCurrentItem()
             {
                lock(this)
                {
                 return m_sCurrentItem;
                }
             }

             /// <summary>
             /// AddItem(): Adds one to the item count - used by the worker thread.
             /// </summary>
             public void AddItem()
             {
                lock(this)
                 m_iItemCount += 1;
             }

             /// <summary>
             /// SetSectionCount(): used to set the number of sections to be processed.
             /// </summary>
             /// <param name="iCount"></param>
             public void SetSectionCount( int iCount )
             {
                lock(this)
                 m_iSectionCount = iCount;
             }

             /// <summary>
             /// SetCurrentItem(): Used by the worker thread to set the current item being processed.
             /// </summary>
             /// <param name="sItem"></param>
             public void SetCurrentItem( String sItem )
             {
                lock(this)
                 m_sCurrentItem = sItem;
             }

             /// <summary>
             /// AddProcessNote(String): Used by the worker thread to
             /// </summary>
             /// <param name="sNote"></param>
             public void AddProcessNote( String sNote )
             {
                lock(this)
                 m_alProcessNotes.Add( sNote );

                if( _nantProject != null )
                 _nantProject.Log(NAnt.Core.Level.Verbose, " Processing... {0}", sNote );
             }

             /// <summary>
             /// Notifies the UI thread of a fatal error that has occurred.
             /// </summary>
             /// <param name="sNote"></param>
             public void SetFatalError( String sNote )
             {
                lock(this)
                 m_alProcessNotes.Add( sNote );

                if( _nantProject != null )
                 _nantProject.Log(NAnt.Core.Level.Error, sNote );
                
             }

             /// <summary>
             /// SetCurrentSection(): Sets the current section name being processed
             /// </summary>
             /// <param name="sSection"></param>
             public void SetCurrentSection( String sSection )
             {
                lock(this)
                {
                 if (_nantProject != null )
                 {
                     _nantProject.Log(NAnt.Core.Level.Info, "Processing Segment ({0} of {1}): {2}", m_iCurrentSection+1, m_iSectionCount ,sSection );
                 }

                 m_sCurrentSection = sSection;
                 m_iCurrentSection += 1;
                }
             }
         }


         /// <summary>
         /// class vssManager: This class is reponsible for managing the interation
         /// with Source Control. Eventually moving this to a base/derived class pair
         /// could allow for supporting other source control systems.
         /// </summary>
         private class vssManager
         {
             private const int VSSITEM_PROJECT = 0;
             private const int VSSITEM_ITEM = 1;

             private StreamWriter m_sw;
             private ProcessState m_oProcessState;

             public vssManager( StreamWriter sw, ProcessState oProcessState )
             {
                m_sw = sw;
                m_oProcessState = oProcessState;
             }


             /// <summary>
             /// ProcessItem(): Processes a given source safe item and adds it to
             /// the output stream. The method deletes the file on the way out to
             /// ensure that subsequent reads of the same file will result in a physical
             /// code get.
             /// </summary>
             /// <param name="item">The sourcesafe item object</param>
             /// <param name="sLocalPath">The local path where the physical file should be stored while processing.</param>
             private void ProcessItem( VSSItem item, String sLocalPath )
             {
                
                 // Check to see if we've been requested to stop.
                if ( m_oProcessState.IsStopped() )
                 return;

                // Add a process warning if the file is checked out
                if( item.IsCheckedOut != 0 )
                {
                 IVSSCheckouts oCO = item.Checkouts;
                 String sName = "";

                 if( oCO.Count > 0 )
                 {
                     IEnumerator oEnum = oCO.GetEnumerator();
                     oEnum.MoveNext();
                     IVSSCheckout oCheck = (IVSSCheckout)oEnum.Current;
                            

                     sName = " to " + oCheck.Username;
                 }
                                            
                 m_oProcessState.AddProcessNote( "WARNING: " + item.Name + " is checked out" + sName + ".");
                }
                    

                // perform a get of the file to the local path
                String sSpec = System.IO.Path.Combine(sLocalPath, item.Name);

                int iFlags = Convert.ToInt32(VSSFlags.VSSFLAG_REPREPLACE) |
                             Convert.ToInt32(VSSFlags.VSSFLAG_USERRONO);

                item.Get(ref sSpec, iFlags);

                m_oProcessState.SetCurrentItem(item.Name);
                m_oProcessState.AddItem();


                // add the contents of the file to the output stream
                using (StreamReader sr = new StreamReader(sSpec))
                {
                 String line;

                 while(( line = sr.ReadLine()) != null)
                     m_sw.WriteLine(line);
                }

                // delete the processed file.
                System.IO.File.Delete(sSpec);

             }

             /// <summary>
             /// ProcessProject(): Recurses into a folder/project hierarchy when given a parent project.
             /// The function dispatches the ProcessItem call when it reaches a file object.
             /// </summary>
             /// <param name="item"></param>
             /// <param name="sLocalPath"></param>
             private void ProcessProject( VSSItem item, String sLocalPath )
             {
                    
                foreach( VSSItem subItem in item.get_Items(false))
                {
                 if( m_oProcessState.IsStopped() )
                     return;

                 if( subItem.Type == VSSITEM_PROJECT )
                     ProcessProject( subItem, sLocalPath );
                 else
                     ProcessItem( subItem, sLocalPath );
                }
             }

             /// <summary>
             /// ProcessLineItem(): Public interface that accepts a path from the client and begins processing the structure.
             /// </summary>
             /// <param name="oVssDb"></param>
             /// <param name="sPath"></param>
             /// <param name="sLocalPath"></param>
             public void ProcessLineItem( VSSDatabase oVssDb, String sPath, String sLocalPath )
             {
                VSSItem item = oVssDb.get_VSSItem(sPath, false);

                if( item.Type == VSSITEM_PROJECT )
                 ProcessProject( item, sLocalPath);
                else
                 ProcessItem( item, sLocalPath );

             }

             /// <summary>
             /// GetSourceSafeDatabase(): Takes the parent XMLDocument and locates the sourcesafe login credentials for processing purposes.
             /// This function returns an active sourcesafe database object used for subsequent processing.
             /// </summary>
             /// <param name="oDoc"></param>
             /// <returns></returns>
             public static VSSDatabase GetSourceSafeDatabase(XmlDocument oDoc)
             {
                String sIniPath, sUserName, sPassword;
                try
                {
                 XmlNodeList nodeIni = oDoc.SelectNodes("/hssSourceManagerSection/vssConfig");

                 sIniPath = nodeIni.Item(0).Attributes.GetNamedItem("IniFile").Value;
                 sUserName = nodeIni.Item(0).Attributes.GetNamedItem("UserName").Value;
                 sPassword = nodeIni.Item(0).Attributes.GetNamedItem("Password").Value;
                }
                catch (Exception ex)
                {
                 throw new Exception("Unable to retrieve SourceSafe configuration information from config file.", ex);
                }


                VSSDatabase oVssDb = new VSSDatabaseClass();

                try
                {
                 oVssDb.Open(sIniPath, sUserName, sPassword);
                }
                catch( Exception ex)
                {
                 throw new Exception("Unable to open SourceSafe Database", ex);
                }


                return oVssDb;
                                
             }

         }


         private ProcessState m_oProcessState;
         private XmlDocument m_oBuildConfig;
         private StreamWriter m_sw;
         private NAnt.Core.Project m_nantProject;
         private String m_sLocalPath;


         /// <summary>
         /// GetProcessState(): Returns the live process state - used by the ui thread to gain access to status info.
         /// </summary>
         /// <returns></returns>
         public ProcessState GetProcessState()
         {
             return m_oProcessState;
         }

         /// <summary>
         /// hssSourceControl(): default constructor
         /// </summary>
         public hssSourceControl()
         {
             m_oProcessState = new ProcessState();
         }

         /// <summary>
         /// ProcessSourceProject(): Responsible for taking the source XmlDocument and processing all the items within it
         /// into the stream output.
         /// </summary>
         /// <param name="oDoc">XmlDocument with the agreed upon format to identify processing of the database project.</param>
         /// <param name="sw">StreamWriter object that contains the aggregated text.</param>
         /// <param name="sLocalPath">Directory path to processing area used to hold file temporarily during build.</param>
         /// <param name="nantProject">Nant project object used to direct status output when called from the Nant environment</param>
         public void ProcessSourceProject( XmlDocument oDoc, StreamWriter sw, string sLocalPath, NAnt.Core.Project nantProject )
         {
             try
             {
                // here, we'll connect to SourceSafe and get ready for processing
                VSSDatabase oVss = vssManager.GetSourceSafeDatabase(oDoc);

                // next, we'll iterate through all the configured lines
                XmlNodeList nodeSubs = oDoc.SelectNodes("/hssSourceManagerSection/hssSourceItem");

                vssManager oMgr = new vssManager(sw, m_oProcessState);
                m_oProcessState.NAntProject = nantProject;
            
                m_oProcessState.SetSectionCount( nodeSubs.Count );

                for( int i = 0; i < nodeSubs.Count; i++)
                {
                 if( m_oProcessState.IsStopped() )
                     return;
                
                 XmlNode xNode = nodeSubs.Item(i);
                 String sTitle = xNode.Attributes.GetNamedItem("Value", "").Value;
                 String sPath = xNode.Attributes.GetNamedItem("Path", "").Value;
                 m_oProcessState.SetCurrentSection(sTitle);
                 oMgr.ProcessLineItem(oVss, sPath, sLocalPath);
                }
             }
             catch( Exception ex)
             {
                string sMsg = string.Format("FATAL ERROR: {0}", ex.Message );
                m_oProcessState.SetFatalError( sMsg );
             }
         }

         public void SetThreadData( XmlDocument oBuildConfig,
             StreamWriter sw,
             NAnt.Core.Project nantProject,
             String sLocalPath)
         {
             m_oBuildConfig = oBuildConfig;
             m_sw = sw;
             m_nantProject = nantProject;
             m_sLocalPath = sLocalPath;
         }

         public void ThreadProc()
         {
             ProcessSourceProject( m_oBuildConfig, m_sw, m_sLocalPath, m_nantProject );

         }


    }
    }
  • I finally have FireFox running flawlessly

    I ran into a problem with FireFox about a month ago where I'd get an error popping up regularly saying: "The procedure entry point ??0NS_ConvertUTF16toUTF8@@QAE in xpcom.dll not found." The problem persisted with the 1.0 release. I finally found a reference on Google stating that I should get rid of the ActiveX plugin. I deleted the plugin and the problem is solved. I really enjoy the browser, especially the tabbed browsing feature (not to mention the security and performance considerations).
  • NAnt-ing my database build

    I finished the process of performing my database build using NAnt (actually NAntContrib).  I dusted off an old class library I used with a .Net plug-in I built for automating the database build and modified it so it could handle the NAnt.Core.Project object for floating messages, warnings and errors. 

    Our database project under VS contains all of our stored procs, functions and trigger scripts (among others). Each script is responsible for testing if the contained object currently exists and takes appropriate steps.  Because the processing is order-critical, I've created an XML file format that enforces creation order and also allows for individual files or entire projects to be processed using a given line. 

    I could have gotten this to run using only my class library instead of NAntContrib's SourceSafe additions (for grabbing the BuildScript), but I'll need some of NAntContribs functionality beyond the database build.  My NAnt script does the following:

    (1) Get the database XML build script out of sourcesafe and store it to a working directory.  This XML build script signifies processing order and supports both individual files and directories. 
    (2) Loads the XML build script in an XMLDocument object.
    (3) Creates a StreamWriter object with the appropriate destination file name.
    (4) Passes the objects to my class library to combine all the database scripts together into one file.
    (5) NAnt then moves the file to the build area on the server

    Here is the NAnt script I'm using to complete the task. I purposely pass the XMLDocument object because I'll eventually create different database build configurations using XSLT to automate the collection of these scripts.  I like the way this has turned out. 

    <script language="C#" mainclass="Transform">
    <references>
    <include name="hssScriptBuilder.dll" />
    </references>
    <imports>
       <import name="System.Xml" />
       <import name="System.Xml.XPath" />
       <import name="System.Xml.Xsl" />
    </imports>

    <code>
    <![CDATA[

    //using System.Xml;

    class Transform
    {
       public static void ScriptMain(Project project)
       {
          // load the BuildConfig file gathered previously from sourcesafe      
          XmlDocument xDoc = new XmlDocument();
          xDoc.Load("./dbbuild/solution/BuildConfig.xml");

          // create the file for the entire script
          StreamWriter sw = new StreamWriter("./dbbuild/full_database_upgrade_script.sql");
          
          // hand the parameters to the class library to gather file contents
          hssScriptBuilder.hssSourceControl scc = new hssScriptBuilder.hssSourceControl();
          scc.ProcessSourceProject( xDoc, sw, "./dbbuild/workarea", project);
       }

    }

    ]]>
    </code>
    </script>

  • SQL Service Broker

    I started lookin at the SQL Service Broker last night after reading John Lam's blog entry on the topic.  This looks really cool as it provides guaranteed messaging between SQL Server Instances.  I wouldn't be surprised if it goes beyond this scope, but that's what I'm looking for. 

     I performed this task on SQL 2000 and blogged about it back in July.  My approach under SQL 2000 involved writing a c# cominterop object that took message data and posted it over a WebService call. The structure of the call was similar to the formatting of MSMQ messages.  I wrote the logic to handle message failure and retries using T-SQL.  The webservice along with the com object was packaged and installed on the SQL server boxes.  The solution has been running nicely since that time.

    It will be nice to rely on the tool for guaranteed message delivery.  Developing this type of functionality is generic and doesn't have a lot of business value by itself. It's great that MS has recognized this need and placed the capability in their product.  MSMQ is a nice tool, but try selling an MSMQ-based solution into mid-market companies with less than first class IT support - it's just not feasible. 

  • NantContrib finally building

    I finally got NantContrib working, what a pain!  This is one of those “don't try this at home experiences”.  I got it working between setting up a series of cubes on SQL Server 2000 using SQL Analysis Services.  I'll have to draw up the steps I took and blog them later.

     

  • .Net Code Profiler Summary

    Tim Weaver put together a nice summary of the current .Net Code Profilers.  This is a nice list and I need to spend some time with these tools.  I've been using DevPartner Studio's tool and I've been happy with it, but I'd like to get a look at the features other tools offer.  Thank's Tim, I'm blogging this take a closer look at the tools over the weekend.
  • off-topic: reforming healthcare with a different approach

    Here's an interesting thought on healthcare reform in light of the upcoming election - make the insurance actuarial business a public institution.  Publish and mandate the actuarial tables used by insurance companies - that way all insurance companies must compete on the same basis.  No more of this “ignore the man behind the curtain“ business.  Whether you're buying insurance for one person or a group of people, all actuarial rates are the same.  Any categorical divisions (smokers, skydivers, pre-existing conditions) must be approved and supported outside of the insurance industry. 

    back to Nant...

     

  • Compiling Nant and NantContrib

    I'm continuing the process of getting Nant and NantContrib on my pc.  NantContrib is an extension to Nant that supports source control, msi creation, and other “platform specific” nant components if you are not aware of this.  The documentation states that NantContrib contains features not rolled into the general Nant rollout yet.

    I still have not successfully compiled NantContrib, however I am finally able to compile Nant.  I found that I have to retrieve the project from sourcecontrol (I'm using TortoiseCVS) and then do the build.  I only got this to work once I used the following CVSROOT setting: :pserver:anonymous@cvs.sourceforge.net:/cvsroot/nant

    I'm trying to understand why the NantContrib build is failing even after pulling it down to my machine using TortoiseCVS. 

    I think it's interesting that someone would go to the trouble of building an automation tool designed to create MSI files, and not put together a distribution.  Why should I have to compile something when I can simply have the distribution bits?  I don't care, nor do I ever WANT to care how Nant works or much less compiles.  I guess that's why they call it OpenSores.  Maybe creating a distribution set could be my contribution to the Nant community if it doesn't violate licensing. Where is MSBuild when you need it on a currently shipping product?

More Posts

Our Sponsors

Free Tech Publications