With small steps...: 2007

Saturday, November 3, 2007

Про C++

Плюсы -- подлый язык. Когда пишешь на C -- ставишь во главу стола производительность, когда пишешь на чем-то более высокоуровневом (будь то Python, или, прости господи, Lisp) -- пляшешь от ясного выражения концепций в конструкциях языка.

Когда же пишешь на плюсах, все время пытаешься усидеть на двух стульях разом (что вернуть -- список или итератор, или указатель? список чего? указателей или объектов? уф... и т.д. и т.п.) :( Может, с опытом это пройдет.

Thursday, September 27, 2007

Managing Oracle Forms CVS tree under Powershell

At work, I have to deal with Oracle Forms, which itself is a tool for creating data access forms in a visual way. This tool stores forms as a binary files (.fmb's), so managing them under ordinary version control system like CVS is a pain, because I can't see what changes were made to form (if any). In spite of that, CVS is my only available option.

Fortunately, Oracle Developer Suite has a tool, frmf2xml which can be used to convert binary FMB file to XML, so, gluing it all together (cvs, frmf2xml and ordinary diff utility) with PowerShell, I have written a script, Compare-CvsForms, which is much like cvs diff command, but works with theese binary forms. Actually, I've also written vdiff.py script, based on difflib, which is much more convenient for comparing such a messy data (XML, huh).

So, here is the Compare-CvsForms.ps1 script (I've put it to my PowerShell profile directory):


Param ([string]$formsFile)

if (cvs status $formsFile |
select-string 'Status: (Locally Modified|Needs Checkout|Needs Patch)')
{
# store working copy in another place
$local:temp_fmb = [System.IO.Path]::ChangeExtension(
 [System.IO.Path]::GetTempFileName(), "fmb")
$local:temp_xml = [System.IO.Path]::ChangeExtension(
 [System.IO.Path]::GetTempFileName(), "xml")
Move-Item -force $formsFile $local:temp_fmb
# produce xml of a working copy
frmf2xml $local:temp_fmb | out-null
if ($LastExitCode -ne 0) {
Echo "ERROR: Cannot convert working copy of form to XML!"
Move-Item -Force $local:temp_fmb $formsFile
return
}
# get repositary copy
cvs update 2>&1 | out-null
#produce xml of a repositary copy
frmf2xml $formsFile | out-null
if ($LastExitCode -ne 0) {
Echo "ERROR: Cannot convert CVS copy of form to XML!"
Move-Item -Force $local:temp_fmb $formsFile
Remove-Item -Force $xml_from
return
}
# compare xmls
$xml_from=$formsFile.substring(0, $formsFile.length-4) + "_fmb.xml"
$xml_to=$local:temp_fmb.substring(0, $local:temp_fmb.length-4) + "_fmb.xml"
vdiff.py -context $xml_from $xml_to
if ($LastExitCode -eq 0) {
$formsFile.toUpper() + ": Working copy does not differ from repositary."
}
Move-Item -Force $local:temp_fmb $formsFile
Remove-Item -Force $xml_to,$xml_from
} else {
$formsFile.toUpper() + ": Working copy is up to date."
}

And vdiff.py (pretty straightforward) script (surely, you should have Python installed, if not - you could just as easily replace this script with ordinary diff utility):


#!/usr/bin/python
import os
import sys
import difflib
import tempfile

program_name = os.path.splitext(os.path.basename(os.path.sys.argv[0]))[0].upper()
browser = "c:\\Program Files\\Internet Explorer\\iexplore.exe"

if len(sys.argv) < 3:
  print "\nUSAGE:\n\n    %s FROM-FILE TO-FILE\n" % program_name
  sys.exit(2)

if sys.argv[1] == "-context":
  context = True
  from_fn = sys.argv[2]
  to_fn = sys.argv[3]
else:
  context = False
  from_fn = sys.argv[1]
  to_fn = sys.argv[2]

try:
  from_lines = open(from_fn).readlines()
except IOError:
  print "%s: Error while reading file -- %2" % (program_name, from_fn)
  sys.exit(1)

try:
  to_lines = open(to_fn).readlines()
except IOError:
  print "%s: Error while reading file -- %2" % (program_name, to_fn)
  sys.exit(1)

differs = False
for x in difflib.context_diff(from_lines, to_lines):
  differs = True
  break

if differs:
  diff = difflib.HtmlDiff(wrapcolumn=70).make_file(from_lines, to_lines, from_fn, to_fn, context)
  diff = diff.replace("<head>",
                      '<head><meta http-equiv="Content-Type" content="text/html; charset=utf-8">')
  tmp = tempfile.mkstemp(".html")
  os.write(tmp[0], diff)
  os.close(tmp[0])
  os.spawnl(os.P_WAIT, browser, ' file://' + tmp[1] + '')
  sys.exit(1)
else:
  sys.exit(0)

Scripting with PowerShell actually is a great deal of fun :)

Friday, June 15, 2007

A little correction to Process-File function

Once after processing my C++ source code tree with Process-File my sources became uncompilable. The problem was caused by redirection operator `>', which transformed source files from plain-old ASCII to Unicode. So there's corrected version which uses Out-File cmdlet instead with encoding specified explicitly:


function Process-File([scriptblock]$script, 
                      $filename, 
                      $encoding = "ascii") {
    $local:temp = New-TempFileName
    Get-Content -Encoding $encoding $filename | `
          Foreach $script | `
          out-file -encoding $encoding $local:temp
    Move-Item -force $local:temp $filename
}

Saturday, April 28, 2007

Proper conversion of Mueller dictionary to Unicode

Note: this article is a (bad) translation of my article on this subject written in russian.

This article describes more proper than Ubuntu (and maybe Debian) current package offers conversion of a Mueller dictionary to Unicode (UTF-8) with consequent conversion to dict format.
There is some reasons for doing this job:

Existing conversion from Mueller to dict has a few first dictionary articles corrupted (the ones that describe dictionary terms). I suppose this was caused by executing conversion script (to-dict) in the wrong locale. It should be executed in KOI8-R, not UTF-8.
More to that, original dictionary contains not just plain KOI8-R text, but SilIPA-encoded word transcriptions too. These transcriptions should be handled separately with "SilIPA-to-Unicode IPA Extensions" conversion.
More to that, the script to-dict itself contains a little bug, which caused some words starting with non-alphabetical character (dictionary terms, other words starting with dash or apostroph -- like 'cause, -shaped) to be excluded from dictionary index at all!

So I've decided to convert dictionary by myself. If you want to convert dictionary using my method, you'll have to download my script correct-translation.pl and corrected script to-dict (corrections -- script text converted to UTF-8 and options --allchars --utf8 in dictfmt call).

So, lets start. First of all, create temporary directory and put there original dictionary and those scripts:

~$ mkdir /tmp/mutf
~$ cd /tmp/mutf/
/tmp/mutf$ wget http://uj2.h15.ru/mutf/correct-translation.pl
...
/tmp/mutf$ wget http://uj2.h15.ru/mutf/to-dict
...
/tmp/mutf$ wget http://some.where/Mueller7GPL.koi
...
/tmp/mutf$ chmod 755 correct-translation.pl to-dict
/tmp/mutf$ ls -l
итого 5608
-rwxr-xr-x  1 andrey andrey     531 2005-07-06 16:20 correct-translation.pl
-rw-r--r--  1 andrey andrey 5714222 2005-07-06 16:23 Mueller7GPL.koi
-rwxr-xr-x  1 andrey andrey    6302 2005-07-06 16:22 to-dict
/tmp/mutf$

Note: sorry, I can't put this scripts to some more "permanent" storage, but I can mail them to you if you can't download them.
Now, we're going to translate dictionary to UTF-8. Firstly, we need to convert dictionary using iconv and then, we need to correct transcriptions with correct-translation.pl:

/tmp/mutf$ cat Mueller7GPL.koi | iconv -f koi8-r -t utf-8 | \
> ./correct-translation.pl > Mueller7GPL.utf && rm -f Mueller7GPL.koi
/tmp/mutf$

Now we will convert dictionary to dict format:

/tmp/mutf$ ./to-dict --src-data Mueller7GPL.utf mueller7.data
Writing the header of mueller7.data..
Formatting data (Mueller7GPL.utf -> mueller7.data)..
.
/tmp/mutf$ ./to-dict --data-dict mueller7.data mueller7 && rm -f mueller7.data
dictfmt: mueller7.data -> mueller7.dict and mueller7.index..
46185 headwords
Compressing mueller7.dict..
.
/tmp/mutf$

Now we just need to install dictionary:

/tmp/mutf$ cp mueller7.* /usr/share/dictd/
/tmp/mutf$ dictdconfig -w &&amp; (killall dictd; dictd)

Resulting article can look like this:

Monday, February 12, 2007

Integrating Emacs kill-ring and Windows clipboard

Since I've started using Emacs on Windows, there was one little problem irritating my mind everyday -- by default, GNU Emacs 22 separates kill-ring and Windows clipboard contents, making it REALLY hard to use clipboard -- you can paste text by only pressing mouse-3 (the wheel) on editor window, and you can copy text only by selecting it with your mouse -- no handy C-y, no integration with kill ring.

So I did a little "investigation". There are two variables controlling mentioned integration -- interprogram-cut-function and interprogram-paste-function (described in emacs online documentation, which see). First of which holds function used to send killed text to clipboard, and the second one holds function used to get fresh clipboard text back from clipboard and put it to kill-ring. More to that, file w32-fns.el in your emacs site-lisp tree contains the next (as well as definitions for those functions -- x-select-text and x-get-selection-value):

(setq interprogram-cut-function 'x-select-text)
(setq interprogram-paste-function 'x-get-selection-value)

But for some reason this file isn't loaded on Emacs startup. So, what should we do is to add some loading code to our dotemacs file (do not forget to wrap-n-protect it with some kind of Win32-only conditional -- `(Windows body ...)' macro in my case):

(Windows
(load "w32-fns"))

Voila! Now text from clipboard goes right to the kill-ring and vice versa.

Sunday, January 7, 2007

Process-File function

Some time ago while scripting (either bash or PowerShell) I've found myself using some pattern too often. It can be expressed as

take a file;
process every line in it;
save file back.

E.g. `sed -i' provides this kind of functionality. This was kind of boring, because you can't just write "cat file.txt | do-something > file.txt", and if you really could, this was too long anyway. So I wrote function which does just that -- takes file, processes it with some code (specified as an argument), and writes it back. Here it goes:

function Process-File([scriptblock]$script, $filename) {
    $local:temp = [System.IO.Path]::GetTempFileName()
    Get-Content $filename | Foreach $script > $local:temp
    Move-Item -force $local:temp $filename
}

Here some testing code:

PS C:\Documents and Settings\andrey> cat test.txt
PS C:\Documents and Settings\andrey> echo "abcabc" > test.txt
PS C:\Documents and Settings\andrey> cat test.txt
abcabc
PS C:\Documents and Settings\andrey> Process-File { $_ -replace 'a','b' } test.txt
PS C:\Documents and Settings\andrey> cat test.txt
bbcbbc
PS C:\Documents and Settings\andrey>

PowerShell is just wonderfull.

With small steps...