Saturday, July 31, 2010

find mysqld configure file position

检查mysqld读取配置文件的位置,可以用下面这个命令:
$> mysqld --verbose --help|grep -A 1 'Default options'

摘自《高性能MySQL》

Tuesday, July 20, 2010

google chrome about command list

about:memory
about:stats
about:network
about:internets
about:histograms
about:dns
about:cache
about:crash
about:plugins
about:version

ubuntu 9.10 上安装facebook XHP

http://wiki.github.com/facebook/xhp/building-xhp

1. Download xhp from http://github.com/facebook/xhp/downloads

$> sudo apt-get install php gcc flex bison re2c

$> phpize
$> ./configure
$> make
$> sudo make install

3. add xhp config file to php
$> sudo vi /etc/php5/conf.d/xhp.ini
# add below line to xhp.ini file
extension=xhp.so

4. test xhp, if you get a syntax error then XHP is not working.
$> php -r 'echo "XHP!\n"; exit; <a />;'

5. xhp configure
xhp.idx_expr = {1 | 0} (default: 0): This is a bonus feature of XHP which adds support for the [] operator on the return value of a function. If you enable this you will be able to write code like foo()['bar'] without hitting syntax errors. This configuration is mainly around to fix a personal annoyance with PHP's grammar, and has nothing to do with XHP's core features.

xhp.include_debug = {1 | 0} (default: 1): You can disable XHP node annotations by turning this setting off. This may in some cases marginally increase performance of XHP-heavy code. It is highly recommended that you leave this on, as tracking errors in XHP applications is very difficult without annotations.

Saturday, July 17, 2010

acme网站上几个http小工具

1. http_load: http://www.acme.com/software/http_load/

同apache ab和JMeter一样可以对数据库和web服务器作压力测试。
$> vi urls


$> ./http_load -rate 5 -seconds 10 urls
49 fetches
2 max parallel
289884 bytes
in 10.0148 seconds 5916 mean bytes/connection 4.89274 fetches/sec
28945.5 bytes/sec msecs/connect: 28.8932 mean
44.243 max
24.488 min msecs/first-response: 63.5362 mean
81.624 max
57.803 min
HTTP response codes: code 200 -- 49

2. http_ping: http://www.acme.com/software/http_ping/

与ping命令类似,但是是通过http请求来检查服务器,而非ICMP请求服务器返回响应。

$> http_ping http://www.example.com/
7816 bytes from http://www.example.com/:
246.602 ms (9.923c/23.074r/213.605d)
7816 bytes from http://www.example.com/:
189.997 ms (11.619c/22.971r/155.407d)
7816 bytes from http://www.example.com/:
190.463 ms (8.994c/25.091r/156.378d)
7816 bytes from http://www.example.com/:
190.07 ms (9.234c/23.9r/156.936d)
7816 bytes from http://www.example.com/:
190.706 ms (10.142c/46.579r/133.985d)
--- http://www.example.com/
http_ping statistics
--- 5 fetches started, 5 completed (100%), 0 failures (0%), 0 timeouts (0%)
total min/avg/max = 189.997/201.568/246.602 ms
connect min/avg/max = 8.994/9.9824/11.619 ms
response min/avg/max = 22.971/28.323/46.579 ms
data min/avg/max = 133.985/163.262/213.605 ms

Tuesday, July 06, 2010

mongodb tutorial and references

Introduction


MongoDB is a collection-oriented, schema-free document database.

By collection-oriented, we mean that data is grouped into sets that are called 'collections'. Each collection has a unique name in the database, and can contain an unlimited number of documents. Collections are analogous to tables in a RDBMS, except that they don't have any defined schema.

By schema-free, we mean that the database doesn't need to know anything about the structure of the documents that you store in a collection. In fact, you can store documents with different structure in the same collection if you so choose.

By document, we mean that we store data that is a structured collection of key-value pairs, where keys are strings, and values are any of a rich set of data types, including arrays and documents. We call this data format "BSON" for "Binary Serialized dOcument Notation."

MongoDB is a server process that runs on Linux, Windows and OS X. It can be run both as a 32 or 64-bit application. We recommend running in 64-bit mode, since Mongo is limited to a total data size of about 2GB for all databases in 32-bit mode.

The MongoDB process listens on port 27017 by default (note that this can be set at start time - please see Command Line Parameters for more information).

MongoDB stores its data in files (default location is /data/db/), and uses memory mapped files for data management for efficiency.

Example data of mongodb


$> bin/mongo --shell slides.js


/*
* Slides for presentation on Mastering the MongoDB Shell.
* Copyright 2010 Mike Dirolf (http://dirolf.com)
* This work is licensed under the Creative Commons
* Attribution-Noncommercial-Share Alike 3.0 United States License. To
* view a copy of this license, visit
* http://creativecommons.org/licenses/by-nc-sa/3.0/us/ or send a
* letter to Creative Commons, 171 Second Street, Suite 300, San
* Francisco, California, 94105, USA.
* Originally given at MongoSF on 4/30/2010.
* Modified and given at MongoNYC on 5/21/2010.
* To use: run `mongo --shell slides.js`
*/
db = db.getSisterDB("shell");
db.dropDatabase();
// some sample data
for (var i = 0; i < 1000; i += 1) {
db.data.save({
x: i
});
} // "slides"
db.deck.save({
slide: 0,
welcome: "to MongoNYC!",
hashtag: "#mongonyc",
mirror: "http://confmirror.10gen.com/"
});
db.deck.save({
slide: 1,
title: "Mastering the MongoDB Shell",
who: "Mike Dirolf, 10gen",
handle: "@mdirolf"
});
db.deck.save({
slide: 2,
question: "what is the shell?",
answer: "a better Powerpoint?"
});
db.deck.save({
slide: 3,
question: "what is the shell?",
answer: "a full JavaScript environment"
});
db.deck.save({
slide: 4,
question: "what is the shell?",
answer: "a reference MongoDB client"
});
db.deck.save({
slide: 5,
"use cases": ["administrative scripting", "exploring and debugging", "learning (and teaching!)"]
});
db.deck.save({
slide: 6,
repl: ["arrows for history", "^L"]
});
db.deck.save({
slide: 7,
"getting help": ["help", "db.help", "db.foo.help"]
});
db.deck.save({
slide: 8,
"show": ["dbs", "collections", "users", "profile"]
});
db.deck.save({
slide: 9,
navigating: "databases",
how: "'use' or 'db.getSisterDB'"
});
db.deck.save({
slide: 10,
navigating: "collections",
how: "dots, brackets, or 'db.getCollection'",
note: "careful with names like foo-bar"
});
db.deck.save({
slide: 11,
"basic operations": ["insert", "findOne", "find", "remove"]
});
db.deck.save({
slide: 12,
"fun with cursors": ["auto-iteration", "it"]
});
db.deck.save({
slide: 13,
"error checking": "auto 'db.getLastError'"
});
db.deck.save({
slide: 14,
"commands": ["count", "stats", "repairDatabase"],
"meta": "listCommands"
});
db.deck.save({
slide: 15,
"pro tip!": "viewing JS source"
});
db.deck.save({
slide: 16,
"getting help": "--help"
});
db.deck.save({
slide: 17,
scripting: "run .js files",
tools: ["--eval", "--shell", "runProgram"]
});
db.deck.save({
slide: 18,
warning: "dates in JS suck"
});
db.deck.save({
slide: 19,
warning: "array iteration in JS sucks"
});
db.deck.save({
slide: 20,
warning: "numeric types in JS suck"
});
db.deck.save({
slide: 21,
so: "why JS?"
});
db.deck.save({
slide: 22,
homework: ["convince 2 friends to try MongoDB", "send feedback @mdirolf"]
});
db.deck.save({
slide: 23,
url: "github.com/mdirolf/shell_presentation",
questions: "?"
}); // current slide
var current = 0; // print current slide and advance
var next = function() {
var slide = db.deck.findOne({
slide: current
});
if (slide) {
current++;
delete slide._id;
delete slide.slide;
print(tojson(slide, null, false));
} else {
print("The End!");
}
}; // go to slide and print
var go = function(n) {
current = n;
next();
}; // repeat the previous slide
var again = function() {
current--;
next();
};


Basic Commands of mongodb



http://www.mongodb.org/display/DOCS/List+of+Database+Commands
http://rezmuh.sixceedinc.com/2010/02/basic-commands-to-get-you-started-with-mongodb.html

Privileged Commands

Certain operations are for the database administrator only. These privileged operations may only be performed on the special database named admin.

Start the server like this:
$> bin/mongod --dbpath /path/to/data

Stop it with Ctrl-C or with kill (but don’t use kill -9, which doesn’t give the server a chance to shut down cleanly and flush data to disk).

Viewing stats without the mongo console:
Visit http://localhost:28017 (28017 is the port the server is running on, plus 1000) to get an overview of what’s going on.
You can also query http://localhost:28017/_status to see the same data that db.serverStatus() returns, but in JSON format.

Use client command console:

$> bin/mongo

> use admin;
> db.addUser('dummy, 'dummy') // create new user with password "dummy"
> db.runCommand("shutdown"); // shut down the database

$> bin/mongo -u dummy -p dummy admin

Create a new Database:

> use new_database
> db.addUser('dummy', 'differentpassword')

$ bin/mongo -u dummy -p differentpassword new_database


Deleting a database:

> use [db name]
> db.dropDatabase()

Working with Collections (Tables):
> show collections

Lists all the available databases:
> show dbs

To get a list of data in a specific collection, use:
> db.[collection name].find()

Directory Global functions and properties:

> for(var o in this) {
print(o);
}
__quiet
chatty
friendlyEqual
doassert
assert
argumentsToArray
isString
isNumber
isObject
tojson
tojsonObject
shellPrint
printjson
shellPrintHelper
shellHelper
help
Random
killWithUris
Geo
connect
MR
MapReduceResult
__lastres__
sleep_
sleep
quit_
quit
getMemInfo_
getMemInfo
_srand_
_srand
_rand_
_rand
_isWindows_
_isWindows
_startMongoProgram_
_startMongoProgram
runProgram_
runProgram
runMongoProgram_
runMongoProgram
stopMongod_
stopMongod
stopMongoProgram_
stopMongoProgram
stopMongoProgramByPid_
stopMongoProgramByPid
rawMongoProgramOutput_
rawMongoProgramOutput
clearRawMongoProgramOutput_
clearRawMongoProgramOutput
removeFile_
removeFile
listFiles_
listFiles
resetDbpath_
resetDbpath
copyDbpath_
copyDbpath
_parsePath
_parsePort
createMongoArgs
startMongodTest
startMongod
startMongodNoReset
startMongos
startMongoProgram
startMongoProgramNoConnect
myPort
ShardingTest
printShardingStatus
MongodRunner
ReplPair
ToolTest
ReplTest
allocatePorts
SyncCCTest
db
hex_md5_
hex_md5
version_
version
i
current
next
__iscmd__
___it___
it

Directory properties of db:

> for(var o in db){
print(o);
}
_mongo
_name
shellPrint
getMongo
getSisterDB
getName
stats
getCollection
commandHelp
runCommand
_dbCommand
_adminCommand
addUser
removeUser
__pwHash
auth
createCollection
getProfilingLevel
dropDatabase
shutdownServer
cloneDatabase
cloneCollection
copyDatabase
repairDatabase
help
printCollectionStats
setProfilingLevel
eval
dbEval
groupeval
groupcmd
group
_groupFixParms
resetError
forceError
getLastError
getLastErrorObj
getLastErrorCmd
getPrevError
getCollectionNames
tojson
toString
currentOp
currentOP
killOp
killOP
getReplicationInfo
printReplicationInfo
printSlaveReplicationInfo
serverBuildInfo
serverStatus
version
printShardingStatus

> db.shell.help()
DBCollection help
db.foo.count()
db.foo.dataSize()
db.foo.distinct( key ) - eg. db.foo.distinct( 'x' )
db.foo.drop() drop the collection
db.foo.dropIndex(name)
db.foo.dropIndexes()
db.foo.ensureIndex(keypattern,options) - options should be an object with these possible fields: name, unique, dropDups
db.foo.reIndex()
db.foo.find( [query] , [fields]) - first parameter is an optional query filter. second parameter is optional set of fields to return.
e.g. db.foo.find( { x : 77 } , { name : 1 , x : 1 } )
db.foo.find(...).count()
db.foo.find(...).limit(n)
db.foo.find(...).skip(n)
db.foo.find(...).sort(...)
db.foo.findOne([query])
db.foo.findAndModify( { update : ... , remove : bool [, query: {}, sort: {}, 'new': false] } )
db.foo.getDB() get DB object associated with collection
db.foo.getIndexes()
db.foo.group( { key : ..., initial: ..., reduce : ...[, cond: ...] } )
db.foo.mapReduce( mapFunction , reduceFunction , <optional params> )
db.foo.remove(query)
db.foo.renameCollection( newName , <droptarget> ) renames the collection.
db.foo.runCommand( name , <options> ) runs a db command with the given name where the 1st param is the colleciton name
db.foo.save(obj)
db.foo.stats()
db.foo.storageSize() - includes free space allocated to this collection
db.foo.totalIndexSize() - size in bytes of all the indexes
db.foo.totalSize() - storage allocated for all data and indexes
db.foo.update(query, object[, upsert_bool, multi_bool])
db.foo.validate() - SLOW
db.foo.getShardVersion() - only for use with sharding

> db.help()
DB methods:
db.addUser(username, password[, readOnly=false])
db.auth(username, password)
db.cloneDatabase(fromhost)
db.commandHelp(name) returns the help for the command
db.copyDatabase(fromdb, todb, fromhost)
db.createCollection(name, { size : ..., capped : ..., max : ... } )
db.currentOp() displays the current operation in the db
db.dropDatabase()
db.eval(func, args) run code server-side
db.getCollection(cname) same as db['cname'] or db.cname
db.getCollectionNames()
db.getLastError() - just returns the err msg string
db.getLastErrorObj() - return full status object
db.getMongo() get the server connection object
db.getMongo().setSlaveOk() allow this connection to read from the nonmaster member of a replica pair
db.getName()
db.getPrevError()
db.getProfilingLevel()
db.getReplicationInfo()
db.getSisterDB(name) get the db at the same server as this onew
db.killOp(opid) kills the current operation in the db
db.printCollectionStats()
db.printReplicationInfo()
db.printSlaveReplicationInfo()
db.printShardingStatus()
db.removeUser(username)
db.repairDatabase()
db.resetError()
db.runCommand(cmdObj) run a database command. if cmdObj is a string, turns it into { cmdObj : 1 }
db.serverStatus()
db.setProfilingLevel(level,) 0=off 1=slow 2=all
db.shutdownServer()
db.stats()
db.version() current version of the server

> help()
HELP
show dbs show database names
show collections show collections in current database
show users show users in current database
show profile show most recent system.profile entries with time >= 1ms
use <db name> set curent database to <db name>
db.help() help on DB methods
db.foo.help() help on collection methods
db.foo.find() list objects in collection foo
db.foo.find( { a : 1 } ) list objects in foo where a == 1
it result of the last line evaluated; use to further iterate

If I insert the same Document twice, it does not raise an Error?



Using PyMongo for example, why does inserting the same document (read same _id) more than once not raise an error? When we need to detect if a document already exists in the database, we could try catching DuplicateKeyError on insert. Below we explicitly insert the document with the same _id twice but the exception is never raised. Why?

try:
doc = { _id: '123123' }
db.foo.insert(doc)
db.foo.insert(doc)
except pymongo.errors.DuplicateKeyError, error:
print("Same _id inserted twice:", error)

The answer is that DuplicateKeyError will only be raised if we do the insert in safe mode i.e. db.foo.insert(doc, safe=True). The reason why we do not see an error raised with most drivers but with MongoDB's interactive shell is because the shell does a safe insert by default whereas, with most drivers, it is the developers choice whether or not to use a safe insert.

MongoDB's interactive shell:
> doc = {_id: 123421, name: 'test'};
{ "_id" : 123421, "name" : "test" }
> db.deck.insert(doc)
> db.deck.insert(doc)
E11000 duplicate key error index: shell.deck.$_id_ dup key: { : 123421.0 }

Backup



http://effectif.com/mongodb/mongo-administration

There are basically two approaches to backing up a Mongo database:

1. mongodump and mongorestore are the classic approach. Dumps the contents of the database to files. The backup is stored in the same format as Mongo uses internally, so is very efficient. But it’s not a point-in-time snapshot.
2. To get a point-in-time snapshot, shut the database down, copy the disk files (e.g. with cp) and then start mongod up again.

Alternatively, rather than shutting mongod down before making your point-in-time snapshot, you could just stop it from accepting writes:

> db._adminCommand({fsync: 1, lock: 1})
{
"info" : "now locked against writes, use db.$cmd.sys.unlock.findOne() to unlock",
"ok" : 1
}

To unlock the database again, you need to switch to the admin database and then unlock it:

> use admin
switched to db admin
> db.$cmd.sys.unlock.findOne()
{ "ok" : 1, "info" : "unlock requested" }


If you don’t switch to the admin database first you’ll get an unauthorized error:

> db._adminCommand({fsync: 1, lock: 1})
{
"info" : "now locked against writes, use db.$cmd.sys.unlock.findOne() to unlock",
"ok" : 1
}

> db.$cmd.sys.unlock.findOne()
{ "err" : "unauthorized" }


You can take a point in time snapshot from a slave just as easily as your master database, which avoids downtime. This is one of the reasons that running a slave is so strongly recommended…

What RAID should I use?



http://www.mongodb.org/display/DOCS/Developer+FAQ

We recommend not using RAID-5, but rather, RAID-10 or the like. Both will work of course.

Replication



Do it (did you read the previous section?). Seriously.

Start your master and slave up like this:

$ mongod --master --oplogSize 500
$ mongod --slave --source localhost:27017 --port 3000 --dbpath /data/slave

When seeding a new slave server from master use the --fastsync option.

You can see what’s going on with these two commands:

> db.printReplicationInfo() # tells you how long your oplog will last
> db.printSlaveReplicationInfo() # tells you how far behind the slave is

If the slave isn’t keeping up, how do you find out what’s going on? Check the mongo log for any recent errors. Try connecting with the mongo console. Try running queries from the console to see if everything is working. Run the status commands above to try and find out which database is taking up resources. If you can’t work it out hop on the IRC channel; Mathias says they’ll be very responsive.

What is the difference between MongoDB and RDBMSs



http://sunoano.name/ws/mongodb.html#faqs

MySQL, PostgreSQL, ...
----------------------
Server:Port
- Database
- Table
- Row

MongoDB
--------
Server:Port
- Database
- Collection
- Document

The concept of server and database are very similar. But the concept of table and collection are quite different. In RDBMSs a table is a rectangle. It is all columns and rows. Each row has a fixed number of columns, if we add a new column, we add that column to each and every row.

In MongoDB a collection is more like a really big box and each document is like a little bag of stuff in that box. Each bag contains whatever it needs in a totally flexible manner (read schema-less). However, schema-less does not equal type-less i.e. it is just that any document has its own schema, which it may or may not share with any other document. In practice it is normal to have the same schema for all the documents in collection.

Clone Database



http://www.mongodb.org/display/DOCS/Clone+Database

MongoDB includes commands for copying a database from one server to another.

// copy an entire database from one name on one server to another
// name on another server. omit <from_hostname> to copy from one
// name to another on the same server.
db.copyDatabase(<from_dbname>, <to_dbname>, <from_hostname>);
// if you must authenticate with the source database
db.copyDatabase(<from_dbname>, <to_dbname>, <from_hostname>, <username>, <password>);
// in "command" syntax (runnable from any driver):
db.runCommand( { copydb : 1, fromdb : ..., todb : ..., fromhost : ... } );
// command syntax for authenticating with the source:
n = db.runCommand( { copydbgetnonce : 1, fromhost: ... } );
db.runCommand( { copydb : 1, fromhost: ..., fromdb: ..., todb: ..., username: ..., nonce: n.nonce, key: <hash of username, nonce, password > } );

// clone the current database (implied by 'db') from another host
var fromhost = ...;
print("about to get a copy of database " + db + " from " + fromhost);
db.cloneDatabase(fromhost);
// in "command" syntax (runnable from any driver):
db.runCommand( { clone : fromhost } );

What might be a use case for --fork?



The --fork switch forks a mongod process of another exiting process. One use case where this is pretty handy for example is if we SSH into a remote machine and start a MongoDB and leave right away after a new mongod got started.

The fastest way to do so would simply be to append /path/to/mongod --fork to the SSH command line since this will skip creating an intermediate shell and bring up a remote mongod right away.

Are there any Reasons not to use MongoDB?



http://sunoano.name/ws/mongodb.html#faqs

1. We need transactions (read ACID).
2. Our data is very relational.
3. Related to 2, we want to be able to do joins on the server (but can not do embedded objects / arrays).
4. We need triggers on our tables. There might be triggers available soon however.
5. Related to 4, we rely on triggers (or similar functionality) to do cascading updates. or deletes. As for #4, this issue probably goes away once triggers are available.
6. We need the database to enforce referential integrity (MongoDB has no notion of this at all).
7. If we need 100% per node durability.

Use Cases



http://www.mongodb.org/display/DOCS/Comparing+Mongo+DB+and+Couch+DB

It may be helpful to look at some particular problems and consider how we could solve them.

* if we were building Lotus Notes, we would use Couch as its programmer versioning reconciliation/MVCC model fits perfectly. Any problem where data is offline for hours then back online would fit this. In general, if we need several eventually consistent master-master replica databases, geographically distributed, often offline, we would use Couch.
* if we had very high performance requirements we would use Mongo. For example, web site user profile object storage and caching of data from other sources.
* if we were building a system with very critical transactions, such as bond trading, we would not use MongoDB for those transactions -- although we might in hybrid for other data elements of the system. For something like this we would likely choose a traditional RDBMS.
* for a problem with very high update rates, we would use Mongo as it is good at that. For example, updating real time analytics counters for a web sites (pages views, visits, etc.)

Generally, we find MongoDB to be a very good fit for building web infrastructure.

ruby driver of mongodb tutorial



http://api.mongodb.org/ruby/1.0.3/index.html
http://www.mongodb.org/display/DOCS/Ruby+Tutorial
http://github.com/mongodb/mongo-ruby-driver

$> gem update --system
$> gem install mongo
$> gem install bson
$> gem install bson_ext

Making a Connection:
# db = Mongo::Connection.new.db("shell")
# db = Mongo::Connection.new("localhost").db("shell")
db = Mongo::Connection.new("localhost", 27017).db("shell")


Listing All Databases:
m = Mongo::Connection.new # (optional host/port args)
m.database_names.each { |name| puts name }
m.database_info.each { |info| puts info.inspect}


Dropping a Database:
m.drop_database('database_name')


Authentication (Optional):
auth = db.authenticate(my_user_name, my_password)


Getting a Collection:
deck = db.collection("deck")


Inserting a Document:
deck = db['deck']

doc = {"name" => "MongoDB", "type" => "database", "count" => 1,
"info" => {"x" => 203, "y" => '102'}}
deck.insert(doc)
100.times { |i| deck.insert("i" => i) }


Finding the First Document In a Collection using find_one():
my_doc = deck.find_one()
puts my_doc.inspect


Note the _id element has been added automatically by MongoDB to your document. Remember, MongoDB reserves element names that start with _ for internal use.

Counting Documents in a Collection:
puts deck.count()


Using a Cursor to get all of the Documents:
deck.find.each { |row| puts row.inspect }


Getting a Single Document with a Query:
deck.find("i" => 71).each { |row| puts row.inspect }


Getting a Set of Documents With a Query:
deck.find("i" => {"$gt" => 50}).each { |row| puts row }


Querying with Regular Expressions:
deck.find("question" => /w/i).each { |row| puts row }


Creating An Index:
To create an index, you specify an index name and an array of field names to be indexed, or a single field name.
deck.create_index("i")
# deck.create_index([["i", Mongo::ASCENDING]])


Getting a List of Indexes on a Collection:
deck.index_information


Ruby driver example


require 'rubygems' not necessary for Ruby 1.9
require 'mongo'
# 需要先导入slides.js之后,再进行下面的ruby代码测试
db = Mongo::Connection.new.db("mydb")
db = Mongo::Connection.new("localhost").db("mydb")
m = Mongo::Connection.new("localhost", 27017)
db = m.db("shell")
puts db.inspect
m.database_names.each { |name| puts name }
m.database_info.each { |info| puts info.inspect}
db.collection_names.each { |name| puts name }
deck = db['deck']
doc = {"name" => "MongoDB", "type" => "database", "count" => 1,
"info" => {"x" => 203, "y" => '102'}}
deck.insert(doc)
100.times { |i| deck.insert("i" => i) }
my_doc = deck.find_one()
puts my_doc.inspect
puts deck.count()
deck.find.each { |row| puts row.inspect }
deck.find("i" => 71).each { |row| puts row.inspect }
deck.find("i" => {"$gt" => 90}).each { |row| puts row }
deck.find("i" => {"$gt" => 20, "$lte" => 30}).each { |row| puts row }
deck.find("question" => /w/i).each { |row| puts row }
deck.create_index([["i", Mongo::ASCENDING]])
p deck.index_information
db.validate_collection('deck')


References: http://www.mongodb.org/display/DOCS/Developer+Zone

Friday, July 02, 2010

Installation XEN 4.0 from source

以下指令使用root用户执行


$> sudo su -

$> apt-get install bcc bin86 gawk bridge-utils iproute libcurl3 libcurl4-openssl-dev bzip2 module-init-tools transfig tgif texinfo texlive-latex-base texlive-latex-recommended texlive-fonts-extra texlive-fonts-recommended pciutils-dev mercurial build-essential make gcc libc6-dev zlib1g-dev python python-dev python-twisted libncurses5-dev patch libvncserver-dev libsdl-dev libjpeg62-dev iasl libbz2-dev e2fslibs-dev git-core uuid-dev

$> hg clone http://xenbits.xen.org/xen-4.0-testing.hg
$> cd xen-4.0-testing.hg

这个指令会下载linux-2.6-pvops.git,需要较长时间
$> make world


这个指令会安装xen.gz和vmlinuz-2.6.31到系统/boot目录下,vmlinuz-2.6.31这个linux kernel文件在下面会重新设置参数并生成,可查看后面linux kernel编译步骤详细说明。
$> make install


不修改.config文件,可以直接重新生成xen, kernels, tools, docs到dist目录中
$> make dist


修改了.config文件,可以用以下指令重新生成kernel
$> make linux-2.6-xen-config CONFIGMODE=menuconfig     (or xconfig)
$> make linux-2.6-xen-build
$> make linux-2.6-xen-install

linux kernel编译步骤

$> git clone git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git linux-2.6-xen

这个指令会从xen.git仓库中下载props dom0 kernel 2.6.31.x,大概是1.6G左右,需要不少时间
$> cd linux-2.6-xen

这个指令的作用是检查源码是否有.o文件和依赖问题,从全新的源码编译时不需要这一步。make mrproper主要清除环境变量及配置文件
$ make mrproper


设置linux kernel编译参数,在图形界面中输入斜杠"/"可以查询"XEN_DOM0"、"PAE"、"HIGHPTE"等参数设置
$> make menuconfig


退出并保存配置文件之后,手工调整.config文件,并且在内核参数设置中需要注意:
1、如果编译32位的内核,XEN需要有PAE支持,(Processor type and features -> High Memory Support (64GB) -> PAE (Physical Address Extension) Support),对于64位内核PAE不需要,32位的操作系统最大内存支持不到4G,安装了PAE之后可以支持64G。
2、编译32位的内核,必须设置CONFIG_HIGHPTE=n。
3、要为DOM0开启ACPI功能。
4、在.config文件中添加以下选项,并重新运行make menuconfig,并检查XEN_DOM0等参数设置状态

CONFIG_ACPI_PROCFS=y
CONFIG_XEN=y
CONFIG_XEN_MAX_DOMAIN_MEMORY=32
CONFIG_XEN_SAVE_RESTORE=y
CONFIG_XEN_DOM0=y
CONFIG_XEN_PRIVILEGED_GUEST=y
CONFIG_XEN_PCI=y
CONFIG_PCI_XEN=y
CONFIG_XEN_BLKDEV_FRONTEND=m
CONFIG_NETXEN_NIC=m
CONFIG_XEN_NETDEV_FRONTEND=m
CONFIG_XEN_KBDDEV_FRONTEND=m
CONFIG_HVC_XEN=y
CONFIG_XEN_FBDEV_FRONTEND=m
CONFIG_XEN_BALLOON=y
CONFIG_XEN_SCRUB_PAGES=y
CONFIG_XEN_DEV_EVTCHN=y
CONFIG_XEN_BACKEND=y
CONFIG_XEN_BLKDEV_BACKEND=y
CONFIG_XEN_NETDEV_BACKEND=y
CONFIG_XENFS=y
CONFIG_XEN_COMPAT_XENFS=y
CONFIG_XEN_XENBUS_FRONTEND=m
CONFIG_XEN_PCIDEV_FRONTEND=y

在RHEL5和CentOS5中,需要在.config文件中加入以下2个参数:

CONFIG_SYSFS_DEPRECATED=y
CONFIG_SYSFS_DEPRECATED_V2=y


要使用initrd,编译内核时必须选择以下两项:

CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_INITRD=y

重新运行menuconfig,检查配置,最后保存配置文件
$> make menuconfig
$> make install

这个指令会将内核映象和相应的System.map拷贝到/boot目录下。

可以查看指令运行完成后,在/boot目录下新增加的文件列表
$> ll --sort=time /boot/
total 214594
-rw-r--r-- 1 root root 109K 2010-07-02 11:51 config-2.6.31.13
-rw-r--r-- 1 root root 1.6M 2010-07-02 11:51 System.map-2.6.31.13
-rw-r--r-- 1 root root 3.9M 2010-07-02 11:51 vmlinuz-2.6.31.13
...


生成初始化镜像文件(initialed ramdisk)
$> cd /boot
$> depmod 2.6.31.13
# ubuntu中生成初如化镜像文件的指令
$> mkinitramfs -o initrd-2.6.31-xen.img 2.6.31.13
# 或者是用这个指令(CentOS5)
$> mkinitrd -v -f --with=aacraid --with=sd_mod --with=scsi_mod initrd-2.6.31-xen.img 2.6.31.13

$> vi /boot/grub/menu.lst

title Xen 4.0, dom0 Linux kernel 2.6.31.13
kernel /boot/xen.gz dom0_mem=512M
module /boot/vmlinuz-2.6.31.13 root=/dev/sda7 ro nomodeset
module /boot/initrd-2.6.31-xen.img


注:一般内核编译中会有以下一些步骤:
$> make menuconfig
$> make
$> make bzImage
$> make modules
$> make modules_install
$> make install


References:
http://wiki.xensource.com/xenwiki/Xen4.0
http://wiki.xensource.com/xenwiki/XenParavirtOps