Monday, September 15, 2008

Colour your Eclipse Chrome!

To add a browser to eclipse, Go to Window -> Preferences -> General -> Web
Browser
and click Add.



To find the location of your chrome installation, try using the Properties of
the Shortcut Link. [Right-Click on Short-Cut -> Properties -> Target]

More chrome command-line options.

Make sure you have the Chrome browser checked to act as your default Eclipse Browser.

Friday, September 12, 2008

Setting SVN with HTTP(S) - It's Simple

I was looking into SVN and it was a great experience setting it up.

svn create --fs-type fsfs <repos-path>
NOTE: Why FSFS is used ?

HOMEWORK before start!!! - For creating new Products and maintain the tags,branches and truck please refer Strategies for Repository Deployment

Taking Backup
  1. Create Dump File
    $ svnadmin dump <repos-path> > <dumpfile>
  2. Import Backup Dump File
    $ svnadmin load <repos-path> < <dumpfile>
Access Protocol: Suggested is HTTPS
  1. Disable svn
    • Make sure the svnserve is not running as deamon or inetd process.
      OR
    • Disable read and write through svnserve - Edit the svnserve.conf and set anon-access=none and auth-access = none
  2. Basic Setup
    • Load Access Modules in httpd.conf
      LoadModule dav_module modules/mod_dav.so
      LoadModule dav_svn_module modules/mod_dav_svn.so
      <Location /repos>
      DAV svn
      SVNPath /absolute/path/to/repository
      </Location>
    • Make sure that the repository has read-write permissions for apache
      chown -R apache <repos-path>
    • Restart server & check

  3. User Authentication
    • Create a htpasswd file with users
      • $ htpasswd -c /path/to/htpasswd/file user - Create User Credential
      • $ htpasswd /path/to/htpasswd/file user - Append Users to List
      • $ htpasswd -b /path/to/htpasswd/file user password - Batch mode, password is visible
    • Add the below to httpd.conf under the SVN Location
    AuthType Basic
    AuthName "Subversion repository"
    AuthUserFile /path/to/htpasswd/file
    # only authenticated users may access the repository
    Require valid-user
    • Restart and check
    NOTE: There is no need to restart server when the htpasswd file changes
  4. Per-Directory Access Control
    • Create an Access control Policy File
      [groups]
      Full-SVN-RO = svnreader
      Proj1-RW = user1 user2 user3
      Proj1-RO = user4
      Proj2-RW = user1 user6 user7
      Proj2-RO = user8 user2
      EveryOne = @Full-SVN-RO, @Proj1-RW, @Proj1-RO, @Proj2-RW, @Proj2-Ro

      # Access control to whole SVN
      [/]
      svnadmin = rw # No Groups allowed to Read-Write the whole SVN Repos
      @Full-SVN-RO = are

      # Access to Proj1
      [/Proj1]
      @Proj1-RW = rw
      @Proj1-RO = r

      # Access to Proj2
      [/Proj2]
      @Proj2-RW = rw
      @Proj2-RO = r

    • Add the below module to httpd.conf
      LoadModule authz_svn_module modules/mod_authz_svn.so
    • Add Access control Policy File in Location of SVN
      AuthzSVNAccessFile /path/to/access/file
    • Restart httpd and check
    NOTE: There is no need to restart server when the Access control Policy File changes
  5. Enable SSL
    • Setup Apache with SSL.
    • Redirect Request to HTTPS
      #Redirect all port 80 requests to 443
      RewriteEngine On
      RewriteCond %{SERVER_PORT} !^443$
      RewriteRule ^/(.*) https://%{SERVER_NAME}/$1 [L,R]
    • Deny if not accessed without SSL. Add the below under the Location of SVN
      SSLRequireSSL
    • Restart and check
  6. Apache logging
    CustomLog logs/svn_logfile "%t %u %{SVN-ACTION}e" env=SVN-ACTION

Single Repository vs. Multiple Repository
I feel Single Repositories are easy to maintain and administer. Only if there are hundred's of project then I suggest Multiple Repositories.

An example of httpd.conf for multiple repositories.
<Location /svn/>
DAV svn
SVNParentPath /data/subv/repos/
</Location>

<Location /svn/testA/>
# Authenticate a user
AuthType Basic
AuthName "TestA Subversion Repository"
AuthUserFile /data/subv-testA/perms/users
Require valid-user
# Authenticate permissions
AuthzSVNAccessFile /data/subv-testA/perms/perms
# Requires SSL
SSLRequireSSL
</Location>

<Location /svn/testB/>
# Authenticate a user
AuthType Basic
AuthName "TestB Subversion Repository"
AuthUserFile /data/subv-testB/perms/users
Require valid-user
# No permissions
# AuthzSVNAccessFile /data/subv-testB/perms/perms
# No SSL
# SSLRequireSSL
</Location>

Thursday, September 04, 2008

Kernel Locking Techniques

link: http://www.linuxjournal.com/node/5833/

With the rise of multi-threaded and multi-process programming, the race condition in many code is becoming evident.
Race Condition
A race hazard (or race condition) is a flaw in a system or process where the output exhibits unexpected critical dependence on the relative timing of events. The term originates with the idea of two signals racing each other to influence the output first.
en.wikipedia.org/wiki/Race_condition
This can be solved by using atomic operations or some locking semantics. Getting to understand which shared data is to locked or made atomic, does an interrupt disturb the locking semantics is tough. If the locking semantics are wrongly designed -- deadend, called deadlock.
Deadlock
In operating systems, a situation in which two or more processes are blocked while waiting for resources that the other controls so that none of the affected processes can continue to execute. Also known as the "deadly embrace."
www.ssuet.edu.pk/taimoor/books/0-7897-1063-3/appa.htm
The linux Kernel provides some functions to reduce data races

Why Do We Need Locking in the Kernel?
Thus the issue here is "relative timing of events" or a synchronization problem. Shared memory, though with it many advantages poses many design issues. A simple i++ (i=2) done by 2 processes will result in an unexpected result 3, when it should have been 4 (increment twice). These critical regions require locking. The Linux kernel provides a family of locking primitives that developers can use to write safe and efficient code.
Critical regions
A critical section is a piece of code that can only be executed by one process at a time. It will usually terminate in fixed time and a process will only have to wait a fixed time to enter it. Some synchronisation mechanism is required at the entry and exit of the critical section to ensure exclusive use, for example a semaphore.
en.wikipedia.org/wiki/Critical_section
Why is SMP Locks in a Uniprocessor Kernel?
Though you may not have a SMP machine when you write code, others who are using your code may. And the linux maintaners, will never insert any code that does not handle locking issues. SMP and UP kernels are distinct. "This allows certain locks not to exist at all in a UP kernel. Different combinations of CONFIG_SMP and CONFIG_PREEMPT compile in varying lock support. It does not matter, however, to the developer: lock everything appropriately and all situations will be covered."

Atomic Operators
Atomic Operations are the simplest ways of achieving kernel synchronization. The other locking primitives are built using Atomic operations.
Atomic Operation
  • An operation that cannot be interrupted docs.cray.com/books/004-2518-002/html-004-2518-002/zglossaryqkdxjklv.html
  • Operation that executes without interference by other concurrently-executing processes or threads. www.cise.ufl.edu/research/ParallelPatterns/PatternLanguage/Background/Glossary.htm
  • Consider the previous example of i++. If we could read i, increment it and write it back to memory in one uninterruptible operation, the race condition discussed above would not be an issue. Atomic operators provide these uninterruptible operations.
    atomic_t v;
    atomic_set(&v, 5); /* v = 5 (atomically) */
    atomic_add(3, &v); /* v = v + 3 (atomically) */
    atomic_dec(&v); /* v = v - 1 (atomically) */
    printf("This will print 7: %d\n", atomic_read(&v));

    Some points to remember.

    • you cannot pass an atomic_t to anything but one of the atomic operators.
    • you cannot pass anything to an atomic operator except an atomic_t.
    Spinlocks
    When more complex locking design is needed, we don't rely on atomic operations. Spinlocks are commonly used in the kernel, defined in include/asm/spinlock.h and include/linux/spinlock.h. The spinlock is a very simple single-holder lock. If a process attempts to acquire a spinlock and it is unavailable, the process will keep trying (spinning) until it can acquire the lock. This simplicity creates a small and fast lock. The basic use of the spinlock is:

    spinlock_t mr_lock = SPIN_LOCK_UNLOCKED;

    unsigned long flags;
    spin_lock_irqsave(&mr_lock, flags);
    /* critical section ... */
    spin_unlock_irqrestore(&mr_lock, flags);

    spin_lock_irqsave() will disable interrupts locally and provide the spinlock on SMP. This covers both interrupt and SMP concurrency issues
    spin_unlock_irqrestore(), interrupts are restored to the state when the lock was acquired.

    With a UP kernel, the above code compiles to the same as:

    unsigned long flags;
    save_flags(flags);
    cli();
    /* critical section ... */
    restore_flags(flags);

    spin_lock_irq() disables and re-enables interrupts unconditionally, in the same manner as cli() and sti(). Use this only when you know that interrupts were not already disabled before the acquisition of the lock.

    spinlock_t mr_lock = SPIN_LOCK_UNLOCKED;
    spin_lock_irq(&mr_lock);
    /* critical section ... */
    spin_unlock_irq(&mr_lock);

    spin_lock() and spin_unlock() methods that acquire and release the specified lock without any interaction with interrupts.
    spin_lock_bh() that implements the standard spinlock as well as disables softirqs. This is needed when you have code outside a softirq that is also used inside a softirq. The corresponding unlock function is naturally spin_unlock_bh().

    You must be careful not to re-acquire a spinlock you already hold, or you will deadlock.

    Spinlocks should be used to lock data in situations where the lock is not held for a long time--recall that a waiting process will spin, doing nothing, waiting for the lock. (See the ``Rules'' Sidebar for guidelines on what is considered a long time.) Thankfully, spinlocks can be used anywhere. You cannot, however, do anything that will sleep while holding a spinlock. For example, never call any function that touches user memory, kmalloc() with the GFP_KERNEL flag, any semaphore functions or any of the schedule functions while holding a spinlock. You have been warned.

    Semaphores
    If you need a lock that is safe to hold for longer periods of time, safe to sleep with or capable of allowing concurrency to do more than one process at a time, Linux provides the semaphore. Semaphores in Linux are sleeping locks. Because they cause a task to sleep on contention, instead of spin, they are used in situations where the lock-held time may be long. Conversely, since they have the overhead of putting a task to sleep and subsequently waking it up, they should not be used where the lock-held time is short.
    Semaphores are represented by a structure, struct semaphore, which is defined in include/asm/semaphore.h. The structure contains a pointer to a wait queue and a usage count. The wait queue is a list of processes blocking on the semaphore. The usage count is the number of concurrently allowed holders. If it is negative, the semaphore is unavailable and the absolute value of the usage count is the number of processes blocked on the wait queue. The usage count is initialized at runtime via sema_init(), typically to 1 (in which case the semaphore is called a mutex).
    Semaphores are manipulated via two methods: down (historically P) and up (historically V). The former attempts to acquire the semaphore and blocks if it fails. The later releases the semaphore, waking up any tasks blocked along the way.
    Semaphore use is simple in Linux. To attempt to acquire a semaphore, call the down_interruptible() function. This function decrements the usage count of the semaphore. If the new value is less than zero, the calling process is added to the wait queue and blocked. If the new value is zero or greater, the process obtains the semaphore and the call returns 0. If a signal is received while blocking, the call returns -EINTR and the semaphore is not acquired.
    The up() function, used to release a semaphore, increments the usage count. If the new value is greater than or equal to zero, one or more tasks on the wait queue will be woken up:

    struct semaphore mr_sem;
    sema_init(&mr_sem, 1); /* usage count is 1 */
    if (down_interruptible(&mr_sem))
    /* semaphore not acquired; received a signal ... */
    /* critical region (semaphore acquired) ... */
    up(&mr_sem);

    down() function, which differs in that it puts the calling task into an uninterruptible sleep. A signal received by a process blocked in uninterruptible sleep is ignored
    down_trylock() function, which attempts to acquire the given semaphore. If the call fails, down_trylock() will return nonzero instead of blocking.

    Reader/Writer Locks
    Since it is typically safe for multiple threads to read data concurrently, so long as nothing modifies the data, reader/writer locks allow multiple concurrent readers but only a single writer (with no concurrent readers). If your data access naturally divides into clear reading and writing patterns, especially with a greater amount of reading than writing, the reader/writer locks are often preferred. The reader/writer spinlock is called an rwlock and is used similarly to the standard spinlock, with the exception of separate reader/writer locking:

    rwlock_t mr_rwlock = RW_LOCK_UNLOCKED;
    read_lock(&mr_rwlock);
    /* critical section (read only) ... */
    read_unlock(&mr_rwlock);
    write_lock(&mr_rwlock);
    /* critical section (read and write) ... */
    write_unlock(&mr_rwlock);

    Likewise, the reader/writer semaphore is called an rw_semaphore and use is identical to the standard semaphore, plus the explicit reader/writer locking:

    struct rw_semaphore mr_rwsem;
    init_rwsem(&mr_rwsem);
    down_read(&mr_rwsem);
    /* critical region (read only) ... */
    up_read(&mr_rwsem);
    down_write(&mr_rwsem);
    /* critical region (read and write) ... */
    up_write(&mr_rwsem);

    Use of reader/writer locks, where appropriate, is an appreciable optimization.
    Note: reader locks cannot be automatically upgraded to the writer variant. Therefore, attempting to acquire exclusive access while holding reader access will deadlock.

    Big-Reader Locks
    Big-reader locks (brlocks), defined in include/linux/brlock.h, are a specialized form of reader/writer locks. It is a spinning lock that is very fast to acquire for reading but incredibly slow to acquire for writing. Therefore, they are ideal in situations where there are many readers and few writers. While the behavior of brlocks is different from that of rwlocks, their usage is identical with the lone exception that brlocks are predefined in brlock_indices (see brlock.h):

    br_read_lock(BR_MR_LOCK);
    /* critical region (read only) ... */
    br_read_unlock(BR_MR_LOCK);

    The Big Kernel Lock
    Linux contains a global kernel lock, kernel_flag, that was originally introduced in kernel 2.0 as the only SMP lock. During 2.2 and 2.4, much work went into removing the global lock from the kernel and replacing it with finer-grained localized locks. Today, the global lock's use is minimal. It still exists, however, and developers need to be aware of it.
    The global kernel lock is called the big kernel lock or BKL. It is a spinning lock that is recursive; therefore two consecutive requests for it will not deadlock the process (as they would for a spinlock). Further, a process can sleep and even enter the scheduler while holding the BKL. When a process holding the BKL enters the scheduler, the lock is dropped so other processes can obtain it. This is also known as release-on-block/sleep These attributes of the BKL helped ease the introduction of SMP during the 2.0 kernel series. Today, however, they should provide plenty of reason not to use the lock.
    Use of the big kernel lock is simple. Call lock_kernel() to acquire the lock and unlock_kernel() to release it. The routine kernel_locked() will return nonzero if the lock is held, zero if not. For example:

    lock_kernel();
    /* critical region ... */
    unlock_kernel();

    Recovering Grub

    I happened to reinstall Win XP on my DebianLinux+WinXP machine and my MBR was overwritten by re-installation.

    I loaded Ubuntu Live CD:
    1. mounted the root parition
    2. chroot to the mount-point
    3. grub-install /dev/hda
    That is it... grub was up and running.