Google App Engine SDK needs python >= 2.6.4

If you're getting "ImportError: No module named _multiprocessing" when using the Google App Engine SDK:

$ ./google-appengine-sdk/dev_appserver.py --debug itrs-test
/home/scottt/work/google-appengine/google_appengine_1.3.2/google/appengine/tools/appcfg.py:41: DeprecationWarning: the sha module is deprecated; use the hashlib module instead
 import sha
/home/scottt/work/google-appengine/google_appengine_1.3.2/google/appengine/tools/dev_appserver_login.py:33: DeprecationWarning: the md5 module is deprecated; use hashlib instead
 import md5
INFO     2010-04-09 21:08:33,956 appengine_rpc.py:159] Server: appengine.google.com
Allow dev_appserver to check for updates on startup? (Y/n):
dev_appserver will check for updates on startup.  To change this setting, edit /home/scottt/.appcfg_nag
INFO     2010-04-09 21:08:37,314 appcfg.py:357] Checking for updates to the SDK.
DEBUG    2010-04-09 21:08:37,317 appengine_rpc.py:345] Sending HTTP request:
POST /api/updatecheck?release=1.3.2&timestamp=1266535890&api_versions=%5B%271%27%5D HTTPS/1.1
Host: appengine.google.com
X-appcfg-api-version: 1
Content-type: application/octet-stream
User-agent: appcfg_py/1.3.2 Linux/2.6.32.10-90.fc12.x86_64 Python/2.6.2.final.0


INFO     2010-04-09 21:08:38,355 appcfg.py:371] The SDK is up to date.
WARNING  2010-04-09 21:08:38,355 datastore_file_stub.py:623] Could not read datastore data from /tmp/dev_appserver.datastore
INFO     2010-04-09 21:08:38,458 dev_appserver_main.py:399] Running application itrs-test on port 8080: http://localhost:8080
DEBUG    2010-04-09 21:08:47,801 dev_appserver.py:488] Matched "/" to CGI dispatcher with path hello.py
DEBUG    2010-04-09 21:08:47,841 dev_appserver.py:1685] Could not import "_multiprocessing": Disallowed C-extension or built-in module
ERROR    2010-04-09 21:08:47,846 dev_appserver.py:3225] Exception encountered handling request
Traceback (most recent call last):
 File "/home/scottt/work/google-appengine/google_appengine_1.3.2/google/appengine/tools/dev_appserver.py", line 3185, in _HandleRequest
   self._Dispatch(dispatcher, self.rfile, outfile, env_dict)
 File "/home/scottt/work/google-appengine/google_appengine_1.3.2/google/appengine/tools/dev_appserver.py", line 3128, in _Dispatch
   base_env_dict=env_dict)
 File "/home/scottt/work/google-appengine/google_appengine_1.3.2/google/appengine/tools/dev_appserver.py", line 515, in Dispatch
   base_env_dict=base_env_dict)
 File "/home/scottt/work/google-appengine/google_appengine_1.3.2/google/appengine/tools/dev_appserver.py", line 2387, in Dispatch
   self._module_dict)
 File "/home/scottt/work/google-appengine/google_appengine_1.3.2/google/appengine/tools/dev_appserver.py", line 2295, in ExecuteCGI
   logging.debug('Executing CGI with env:\n%s', pprint.pformat(env))
 File "/usr/lib64/python2.6/logging/__init__.py", line 1459, in debug
   root.debug(*((msg,)+args), **kwargs)
 File "/usr/lib64/python2.6/logging/__init__.py", line 1018, in debug
   self._log(DEBUG, msg, args, **kwargs)
 File "/usr/lib64/python2.6/logging/__init__.py", line 1142, in _log
   record = self.makeRecord(self.name, level, fn, lno, msg, args, exc_info, func, extra)
 File "/usr/lib64/python2.6/logging/__init__.py", line 1117, in makeRecord
   rv = LogRecord(name, level, fn, lno, msg, args, exc_info, func)
 File "/usr/lib64/python2.6/logging/__init__.py", line 272, in __init__
   from multiprocessing import current_process
 File "/home/scottt/work/google-appengine/google_appengine_1.3.2/google/appengine/tools/dev_appserver.py", line 1272, in Decorate
   return func(self, *args, **kwargs)
 File "/home/scottt/work/google-appengine/google_appengine_1.3.2/google/appengine/tools/dev_appserver.py", line 1922, in load_module
   return self.FindAndLoadModule(submodule, fullname, search_path)
 File "/home/scottt/work/google-appengine/google_appengine_1.3.2/google/appengine/tools/dev_appserver.py", line 1272, in Decorate
   return func(self, *args, **kwargs)
 File "/home/scottt/work/google-appengine/google_appengine_1.3.2/google/appengine/tools/dev_appserver.py", line 1824, in FindAndLoadModule
   description)
 File "/home/scottt/work/google-appengine/google_appengine_1.3.2/google/appengine/tools/dev_appserver.py", line 1272, in Decorate
   return func(self, *args, **kwargs)
 File "/home/scottt/work/google-appengine/google_appengine_1.3.2/google/appengine/tools/dev_appserver.py", line 1775, in LoadModuleRestricted
   description)
 File "/usr/lib64/python2.6/multiprocessing/__init__.py", line 83, in 
   import _multiprocessing
ImportError: No module named _multiprocessing

Your options are:

1. Apply this patch by hand
2. Upgrade to python >= 2.6.4

See http://bugs.python.org/issue7120 for the details.

Ftrace Tutorials

LWN.net Ftrace Articles by Steven Rostedt

• Debugging the kernel using Ftrace - part 1
• Debugging the kernel using Ftrace - part 2
• Secrets of the Ftrace function tracer

 Ever wanted to see what functions are called in a running Linux kernel?

  

    [tracing]# cd /sys/kernel/debug/tracing
    [tracing]# echo function_graph > current_tracer 
    [tracing]# cat trace | head -20
    # tracer: function_graph
    #
    # CPU  DURATION                  FUNCTION CALLS
    # |     |   |                     |   |   |   |
     1)   1.015 us    |        _spin_lock_irqsave();
     1)   0.476 us    |        internal_add_timer();
     1)   0.423 us    |        wake_up_idle_cpu();
     1)   0.461 us    |        _spin_unlock_irqrestore();
     1)   4.770 us    |      }
     1)   5.725 us    |    }
     1)   0.450 us    |    mutex_unlock();
     1) + 24.243 us   |  }
     1)   0.483 us    |  _spin_lock_irq();
     1)   0.517 us    |  _spin_unlock_irq();
     1)               |  prepare_to_wait() {
     1)   0.468 us    |    _spin_lock_irqsave();
     1)   0.502 us    |    _spin_unlock_irqrestore();
     1)   2.411 us    |  }
     1)   0.449 us    |  kthread_should_stop();
     1)               |  schedule() {

KGDB Tutorial

Building a Kernel that supports KGDB

• Upstrteam documentation:  Using kgdb and the kgdb Internals
• Set CONFIG_KGDB=y in your kernel .config (linux-2.6/lib/Kconfig.kgdb):

menuconfig KGDB

        bool "KGDB: kernel debugging with remote gdb"

        depends on HAVE_ARCH_KGDB

        depends on DEBUG_KERNEL && EXPERIMENTAL

        help

          If you say Y here, it will be possible to remotely debug the

          kernel using gdb.  

• Set CONFIG_KGDB_CONSOLE=y to serve as the KGDB I/O driver (linux-2.6/lib/Kconfig.kgdb, linux-2.6/drivers/serial/kgdboc.c):

config KGDB_SERIAL_CONSOLE

        tristate "KGDB: use kgdb over the serial console"

        select CONSOLE_POLL

        select MAGIC_SYSRQ

        default y

        help

          Share a serial console with kgdb. Sysrq-g must be used

          to break in initially.

KGDB Boot and Module Options

1. Boot with kgdboc=<tty-device>,[baud] (ex: kgdboc=ttyAMA1 for qemu-system-arm)
2. (alternatively) From sysfs
   echo TTY_DEVICE > /sys/module/kgdboc/parameters/kgdboc

Connecting GDB to the Kernel through QEMU Emulated Serial Port

• [arm-cross: linux-2.6]$ qemu-system-arm -nographic -s -M integratorcp -kernel
  ./zImage-2.6.32-integratorcp-v5 -serial tcp:localhost:2345,server -net
  nic,vlan=0 -net tap,vlan=0,ifname=tap0,script=./scripts/qemu-ifup
  -append "console=ttyAMA0 root=/dev/nfs
  nfsroot=172.20.0.1:/nfsroot/box,nfsvers=3 rw
  ip=172.20.0.2::172.20.0.1:255.255.255.0 kgdboc=ttyAMA0 kgdbwait"
  
• [arm-cross: linux-2.6]$ telnet localhost:2345
• Wait till the telnet session show "kgdb: Waiting for connection from remote gdb..." then terminate telnet with CTRL-] then CTRL-D
  
• [arm-cross: linux-2.6]$ gdb ./vmlinux
• (gdb) target remote localhost:2345
• Trouble Shooting: (gdb) set debug remote 1

Connecting GDB to the Kernel through a Physical Serial Port

• [scottt@2530p linux-2.6]$ gdb ./vmlinux

  (gdb) set remotebaud 115200

  (gdb) target remote /dev/ttyS0

Fedora 12: Favorite Bugs and Features

Thanksgiving in the U.S. is just around the corner and like clockwork we have another Fedora Linux release.

Fedora 12: Unite

• Favorite (fixed) bug: [531419] qemu issue with non-virtio NICs receiving heavy traffic volumes:
  
  I helped track down and fix this bug upstream, asked nicely and got the patch into Fedora 12's qemu-0.11 package in time. This way people following my How to handle a Linux BSP tutorial can still get working NFS root on an emulated ARM platform.



• Favorite new feature: wireshark USB traffic capture:
  
  The required kernel(usbmon), libpcap and wireshark changes are all in place in Fedora 12 and we can capture USB packets larger then 32 bytes, save them in .pcap files and dissect them in the familiar wireshark user interface.
  
  Everything works out of the box just by "yum install wireshark && wireshark"! Doing USB work have never been more pleasant.

Vrapper: Eclipse Plugin for VIM Addicts

People who were exposed to the VIM editor at a young age tend to become addicted to its keybindings. I recently discovered the Vrapper Eclipse plugin that although still young and not as featureful as some of the plugins for other IDEs listed here made using Eclipse a much more pleasurable experience.

The usual reason a heavy VIM user would consider using Eclipse is for its Java refactoring or C/C++ code browsing, "find all call sites of function F" features. The C++ parsing and code completion part in particular is something proprietary tools like Source Insight does a lot better then older open source tools like cscope.

In a previous job, I hacked extensively on a code base where Java code with deep class hierarchies and C and C++ code from different vintage and style was linked into a single Linux process. Some of that code implemented device drivers in userspace using a chip vendor supplied abstraction layer which is of course not as well designed as the Linux kernel driver API. I relied on Eclipse and cscope + VIM to navigate the Java and C/C++ part of that code base respectively. I wish I could have used the Vrapper plugin then.

How to Handle a Linux BSP: from u-boot to "Hello World!"

1. Get a minimal Linux system capable of running a "Hello World" C application from the pack of software that your chip vendor calls their Linux "Board Support Package".
2. Configure and build the u-boot bootloader, the Linux kernel, the busybox minimal application environment and getting dynamically linked applications working.
3. Uses the qemu emulator to emulate an ARM hardware platform that loads all software over the network.

mozrunner on Fedora x86_64

If you try to use the mozrunner python library on Fedora or Red Hat x86_64, you'll get:

$ mozrunner
Traceback (most recent call last):
  File "/home/scottt/work/itrs_test/env/bin/mozrunner", line 8, in 
    load_entry_point('mozrunner==1.3.5', 'console_scripts', 'mozrunner')()
  File "/home/scottt/work/itrs_test/env/lib/python2.6/site-packages/mozrunner/__init__.py", line 86, in main
    moz = get_moz_from_settings(settings)
  File "/home/scottt/work/itrs_test/env/lib/python2.6/site-packages/mozrunner/__init__.py", line 165, in get_moz_from_settings
    cmd_args=settings['MOZILLA_CMD_ARGS'])
  File "/home/scottt/work/itrs_test/env/lib/python2.6/site-packages/mozrunner/__init__.py", line 131, in get_moz
    raise Exception ('No default or local profile has been set.')
Exception: No default or local profile has been set.

the solution: see mozrunner issue 10 for a trivial patch to ask mozrunner to look under /usr/lib64 instead of just /usr/lib.

If you're trying to use the windmill web testing framework, you'll need mozrunner-1.x-fedora-x86_64.patch instead until someone ports windmill to mozrunner-2.

Don't you just love 64 bit userspace packaging differences (multilib) between Linux distributions?