Sunday, December 19, 2010

spring的属性编辑器CustomDateEditor及日期对象转化

在spring mvc的Controller中,属性在通过依赖注入(DI)时,普通数据类型都能够辨识。但诸如Date之类,就需要自定义属性编辑器解决。否则报如下错误:
org.springframework.beans.TypeMismatchException:
Failed to convert property value of type [java.lang.String] to required type
[java.util.Date] for property 'date'; nested exception is java.lang.IllegalArgumentException: Cannot convert value of type [java.lang.String] to required type [java.util.Date] for property 'date': no matching editors or conversion strategy found
这表示spring无法找到合适的转换策略,需要自己写一个转换器,在spring中称之为属性编辑器。
spring中的属性编辑器可以将字符串转换为相应的对象,然后注入到其它对象中去。
编写自己的属性编辑器的步骤很简单,属性编辑器类需要从java.beans.PropertyEditorSupport类继承,在这个类中有一个setAsText方法,这个方法有一个String类型的参数,通过这个方法,可以将String类型的参数值转换成其他类型的属性。在这个方法中我们还需要使用一个setValue方法,就来指定转换后的对象实例。
spring 中有个 CustomDateEditor 的类就是继承 PropertyEditorSupport 的一个属性编辑器,在Controller中添加一个@InitBinder的Annotation到某个方法上,在方法中指明日期字符串的格式,就可以将符合此格式的字符串转化为日期对象,代码如下:


/**
* <pre>
* HTML forms work with string values only, when your Authority is a complex bean. You need to configure a PropertyEditor to perform conversion between Authority and String:
*
* @InitBinder
* public void initBinder(WebDataBinder b) {
* b.registerCustomEditor(Authority.class, new AuthorityEditor());
* }
*
* private class AuthorityEditor extends PropertyEditorSupport {
* @Override
* public void setAsText(String text) throws IllegalArgumentException {
* // 另外一个例子是根据字符串,从数据库中查找返回对象
* setValue(authorityService.findById(Long.valueOf(text)));
* }
*
* @Override
* public String getAsText() {
* return ((Authority) getValue()).getId();
* }
* }
* </pre>
*
* 这个方法用来将页面表单上传的Date字符串转化成java的Date对象
*
* @param binder
*/
@InitBinder
public void initBinder(WebDataBinder binder) {
DateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd");
dateFormat.setLenient(false);
binder.registerCustomEditor(Date.class, new CustomDateEditor(dateFormat, false));
}


Reference: how to pass a date into bean property

comments bug on firefox


<body>
<!--

这段HTML注释代码因为中间多了2个连字符,在firefox中会导致解析出错,这段内容被做为正常的文本内容显示在浏览器中。
This entire comment -- will show in web browser

-->
</body>

Reference: (SGMLComment) Mozilla interprets a -- (two dashes in a row) inside of a comment or an include improperly

javascript中array方法调用返回window对象

array中的很多方法通过call和apply调用时会返回window对象,如下写法在Firefox、Chrome等浏览器中会取到window对象:


window === ([]).sort.call();
window === ([]).reverse.call();
([]).concat.call()[0] === window


可以将这些array的方法重写,避免它在运行时的this指向window,如重写sort方法:

Array.prototype.sort = (function(sort) { return function(callback) {
return (this == window) ? null : (callback ? sort.call(this, function(a,b) {
return callback(a,b)}) : sort.call(this));
}})(Array.prototype.sort);

Saturday, November 27, 2010

用maven来运行一个main方法或者启动Server

在maven项目的pom.xml文件的plugins中加入"exec-maven-plugin"这个插件,这个在运行"mvn package"时,会在当前的mvn进程中直接执行指定的class文件的main方法,也可以配置其他的参数,让此main在另一个java进程中启动。如果其中将phase的内容改为"test",就会在运行"mvn test"时执行main方法,也可以在命令行里直接用mvn运行,如下注释说明。
更详细的信息和配置方法,可参考http://mojo.codehaus.org/exec-maven-plugin/usage.html说明。

<!-- commandline: mvn exec:java -Dexec.mainClass="org.phpfirefly.test.Server" -->
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.1</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>java</goal>
</goals>
</execution>
</executions>
<configuration>
<mainClass>org.phpfirefly.test.Server</mainClass>
</configuration>
</plugin>

maven2 repository server install

以下为安装免费版本的nexus maven2服务器和简单的设置过程:


$> wget http://nexus.sonatype.org/downloads/nexus-oss-webapp-1.8.0-bundle.tar.gz
$> tar zxvf nexus-oss-webapp-1.8.0-bundle.tar.gz
$> mv nexus-webapp-1.8.0-bundle /usr/local/nexus
$> cd /usr/local/nexus
$> ls bin/jsw/
$> bin/jsw/linux-x86-32/nexus start
$> tail -f logs/wrapper.log

启动服务后默认URL为:http://localhost:8081/nexus
默认的登录名和密码:admin/admin123
nexus默认是关闭远程索引下载功能的,主要是担心会造成对服务器的巨大负担,需要我们手工开启。
开启的方式: 点击Administration菜单下面的Repositories,将这4个仓库Apache Snapshots,Google code,Codehaus Snapshots,Maven Central的Configuration - Download Remote Indexes修改为true。然后在这三个仓库上分别右键,选择reIndex,这样Nexus就会去下载远程的索引文件。
部署构件至Nexus: Nexus提供了两种方式来部署构件,你可以从UI直接上传,也可以配置Maven部署构件,在上传一个版本时,可以将jar包和source jar包一起上传。

本地maven仓库配置文件,默认位置在用户根目录下的".m2"目录,文件名为settings.xml,如果没有,则创建一份,更新内容如下:

<?xml version="1.0" encoding="UTF-8"?>

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">

<pluginGroups>
<!-- pluginGroup
| Specifies a further group identifier to use for plugin lookup.
<pluginGroup>com.your.plugins</pluginGroup>
-->
<pluginGroup>org.mortbay.jetty</pluginGroup>
</pluginGroups>

<profiles>
<profile>
<id>nexus</id>
<repositories>
<repository>
<id>nexus</id>
<name>local nexus</name>
<url>http://localhost:8081/nexus/content/groups/public/</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>nexus</id>
<name>local nexus</name>
<url>http://localhost:8081/nexus/content/groups/public/</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>false</enabled></snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
</profiles>
<activeProfiles>
<activeProfile>nexus</activeProfile>
</activeProfiles>
</settings>

References: http://juvenshun.javaeye.com/blog/349534
http://wj98127.javaeye.com/blog/306358

Friday, November 05, 2010

get Xpath of html element

在Firefox中有二个非常有用的插件,可以直接得到HTML页面中元素的XPath:

XPath Checker - suggests XPath and can be used to test XPath results.
Firebug - XPath suggestions are just one of the many powerful features of this very useful add-on.

XPath的参考资料:

W3Schools XPath Tutorial
W3C XPath Recommendation
XPath Tutorial - with interactive examples

Sunday, October 24, 2010

Scripting java using rhino and javascript

通过Rhino可以使javascript调用java的标准类库,这使得javascript的编程能力得以强化,在服务器端已经有javascript的MVC框架,一般也是基于Rhino的.
首先在ubuntu上可以通过命令安装rhino:
$> sudo apt-get install rhino

安装完成后,在命令行中输入js或者rhino:

$> js
$> rhino
Rhino 1.7 release 2 2010 09 15
js> new Date()
Sun Oct 24 2010 17:24:27 GMT+0800 (CST)
js> /^\d+$/.test("369");
true

就可以进入javascript的控制台.

另外一种方式就是从网上下载rhino 1.5R4.1版本,在命令行中用java运行,下面的代码在命令行的rhino1.7版本中不能运行,因为无法导入java.awt.*的包:
$> java -jar rhino-1.5R4.1.jar
Rhino 1.5 release 4.1 2003 04 21
js> importPackage(java.awt);
js> frame = new Frame("JavaScript")
js> frame.show()
js> button = new Button("OK")
js> frame.add(button)
js> frame.show()
js> function printDate() { print(new Date()) }
js> o = { actionPerformed: printDate }
js> buttonListener = java.awt.event.ActionListener(o)
js> button.addActionListener(buttonListener)

javascript访问java的包和class

java中所有的代码都是class包装了的,而class又是在package下的,rhino为此封装了一个全局对象Packages,通过Packages可以引入所有的java类,如Package.java.lang, Packages.java.io等:

js> Packages.java.io.File
[JavaClass java.io.File]
js> importPackage(java.io)
js> File
[JavaClass java.io.File]

importPackage(java.io)的效果类似java代码中的import java.io.*;
java中的第三方类库也可以通过importClass和importPackage引入,如:
js> importPackage(Packages.org.mozilla.javascript);
js> Context.currentContext
org.mozilla.javascript.Context@1bc887b
js> importClass(java.awt.List)
js> List
[JavaClass java.awt.List]

在引入了java的类库之后,就可以在javascript中使用这些类库了,如:

js> new java.util.Date()
Sun Oct 24 17:46:50 CST 2010
js> new Date()
Sun Oct 24 2010 17:46:54 GMT+0800 (CST)
js> var f = new java.io.File("/etc/hosts");
js> f.exists();
true
js> f.getName();
hosts
js> java.lang.Math.PI
3.141592653589793
js> java.lang.Math.cos(0)
1
js> for (i in f) { print(i) }
getAbsoluteFile
setReadOnly
listFiles
setReadable
writable
hashCode
wait
setExecutable
usableSpace
file
canonicalPath
getUsableSpace
notifyAll
equals
getParent
mkdirs
parent
class
compareTo
freeSpace
getTotalSpace
createNewFile
toString
toURI
toURL
getCanonicalPath
getCanonicalFile
canonicalFile
renameTo
getParentFile
executable
getFreeSpace
absolute
deleteOnExit
canWrite
name
notify
path
canRead
getPath
delete
length
getClass
readable
totalSpace
absoluteFile
lastModified
absolutePath
isAbsolute
list
mkdir
setWritable
isHidden
readOnly
canExecute
isDirectory
hidden
directory
isFile
getName
getAbsolutePath
exists
parentFile
setLastModified
在上面列出的File的方法中,还包括了其从java.lang.Object中继承的所有方法.对于java的重载方法,javascript调用需要用特别的方式.

用javascript实现java的接口

如要实现Runnable接口,按以下方式操作:
js> var obj = { run: function () { print("\nrunning"); } }
js> obj.run()

running
js> var r = new java.lang.Runnable(obj);
js> r.getClass()
class adapter1
js> var t = new java.lang.Thread(r)
Thread[Thread-1,5,main]
js> t.start();
js>
running

用javascript创建java的数组对象

一般直接用javascript创建数组就可以,转为java对象时,rhino会处理类型转换,也可以用以下方法直接创建java数组:
js> var arr = java.lang.reflect.Array.newInstance(java.lang.String, 5); arr[0] = arr[1] = arr[2] = arr[3] = arr[4] = 'create java array using javascript.'
create java array using javascript.
js> arr
[Ljava.lang.String;@1e97f9f
js> arr[1]
create java array using javascript.
js>

Reference: http://www.mozilla.org/rhino/scriptjava.html

Saturday, October 23, 2010

在VIM中如何使用Tidy为HTML验证并格式化内容

在ubuntu中安装tidy只要输入命令:
$> sudo apt-get install tidy

$> man tidy

查看手册,可以看到比较有用的参数设置,一般我的文件都是utf8编码的,所以tidy命令会跟上"-utf8",并且一般情况就是将tidy输出的文件直接替换掉原来的文件,所以"-m"选项也很有用,输出的HTML也希望是格式化,有缩进的代码,这时就要加上"-i"参数,这样在命令行上我的tidy命令会如下所示:

$> tidy -m -i -utf8 target-files1 target-files2

可以将常用的参数设置写在一个配置文件中,在调用tidy命令时指定"-config config-file-name",下面这份配置就是官网上的一个配置例子,稍做了点调整:

// sample config file for HTML tidy
indent: auto
indent-spaces: 4
wrap: 120
markup: yes
output-xml: no
input-xml: no
show-warnings: yes
numeric-entities: yes
quote-marks: yes
quote-nbsp: yes
quote-ampersand: no
break-before-br: no
uppercase-tags: no
uppercase-attributes: no
char-encoding: utf8
new-inline-tags: cfif, cfelse, math, mroot,
mrow, mi, mn, mo, msqrt, mfrac, msubsup, munderover,
munder, mover, mmultiscripts, msup, msub, mtext,
mprescripts, mtable, mtr, mtd, mth
new-blocklevel-tags: cfoutput, cfquery
new-empty-tags: cfelse
在VIM中,如果需要对当前文件进行HTML格式化操作,那么可以在VIM命令模式下输入以下指令:

:!tidy -m -i -utf8 %

References: Clean up your Web pages with HTML TIDY

Tuesday, October 19, 2010

VIM正则表达式说明

Vim 中查找替换使用的正则表达式与一般的编程语言如javascript/ruby/perl不一样,尤其是在进行非贪婪匹配时,下面主要是分析在VIM中怎么进行非贪婪匹配的.

在VIM手册中有这样的说明: If a character is taken literally or not depends on the 'magic' option and the items mentioned next.
It is recommended to always keep the 'magic' option at the default setting, which is 'magic'. This avoids portability problems.

就是说默认的搜索替换是按magic模式进行查找的,如果有字符用反斜杠"\"转义的话,是会在magic模式中检查其对应转义后含意.并且在VIM配置文件中尽量保持使用magic模式.

VIM中有4种正则匹配模式,可以用":h /magic"查看帮助文档的说明,简单说就是有"\m", "\M", "\v", "\V"这四种,最常用的是默认的"\m",其次"\v"在进行分组捕获时也非常有用,因为不需要像在"\m"模式下那样对小括号进行转义,写起来比较麻烦,另二个模式用得较少些.

在帮助手册中对"\v"的说明如下:
Use of "\v" means that in the pattern after it all ASCII characters except '0'-'9', 'a'-'z', 'A'-'Z' and '_' have a special meaning. "very magic"

也就是除了0-9, a-z, A-Z和下划线"_"之外的其他ASCII字符都有特殊含义,如(, ), |, $, ^, [, ], {, }, :, !, ., *, ?, +, <, >等,这与其他的如javascript的正则更接近一些,在这些字符用得比较多的时候,就考虑用"\v"模式,可以少打好多反斜杠,更像其他语言中的正则表达式,更容易看得明白,所以"\v"模式很好用.

关于"\m", "\v"二种模式的比较说明,摘自帮助手册,移除了"\M", "\V"部分,只要掌握好前面的二种模式就已经够用.


\v \m matches
$ $ matches end-of-line
. . matches any character
* * any number of the previous atom
() \(\) grouping into an atom
| \| separating alternatives
\a \a alphabetic character
\\ \\ literal backslash
\. \. literal dot
\{ { literal '{'
a a literal 'a'

对于这二种模式而言,"\a"都是代表字母,其中需要注意的是"\m"中的"|"(或分隔符)需要转义,这个与linux中的grep命令一样,另外其 "(", ")", "{" 这三个字符也需要转义,但是对于"\m"模式, "}"却可以无需转义(The } may optionally be preceded with a backslash: \{n,m\}),所以在"\m"模式中写出来的没有"\v"中的正则表达式更加整齐好明白,下面提到的非贪婪匹配正是与 "{", "}"这对花括号有关.

非贪婪匹配写法


VIM中的匹配1个或者更多相同字符的"+",在magic模式下需要转义,即用"\+"表示.如"\w\+"匹配一个或者一个以上的字母数字或者下划线."\+"和"*"一样是贪婪匹配的.

可以通过":h non-greedy"查看非贪婪匹配的写法.只能在"\m"模式下进行非贪婪匹配,使用".\{-}"进行最小匹配,如果写整齐点也可以用".\{-\}"来表示,如"pa.\{-\}n"可以匹配到"pattern"或者是"pan".摘录帮助手册中关于贪婪/非贪婪匹配说明如下(magic模式):

\{n,m} Matches n to m of the preceding atom, as many as possible
\{n} Matches n of the preceding atom
\{n,} Matches at least n of the preceding atom, as many as possible
\{,m} Matches 0 to m of the preceding atom, as many as possible
\{} Matches 0 or more of the preceding atom, as many as possible (like *)
\{-n,m} Matches n to m of the preceding atom, as few as possible
\{-n} Matches n of the preceding atom
\{-n,} Matches at least n of the preceding atom, as few as possible
\{-,m} Matches 0 to m of the preceding atom, as few as possible
\{-} Matches 0 or more of the preceding atom, as few as possible

匹配包括换号符在内的任意字符


如果匹配内容有换行符,用通配符"."不能匹配换行符,在其他语言如perl/php/javascript是用"/m"修饰符让"."可以匹配字符串内的换行符,VIM中是用一个转义的下划线"\_"加上"."组成的"\_."来表示包括换行符在内的任意字符,与此效果相似的还有"\_^", "\_$", "\_s".关于VIM和perl的正则表达式区别可以通过":h perl-patterns"查看更详细的说明.

字符类(Character classes)


摘录部分常用的字符类如下,这部分多数与其他语言相似,其他很多字符类与别的语言中的字符类完全不一样,并且大小写的字符集不是取反的字符集,如"\i",就不作记录说明.另外如果需要忽略大小写,可查看":h /ignorecase", 在任何位置加入"\c"标记开始忽略字母的大小写.

\s whitespace character: <Space> and <Tab>
\S non-whitespace character; opposite of \s
\d digit: [0-9]
\D non-digit: [^0-9]
\w word character: [0-9A-Za-z_]
\W non-word character: [^0-9A-Za-z_]
\a alphabetic character: [A-Za-z]
\A non-alphabetic character: [^A-Za-z]
\l lowercase character: [a-z]
\u uppercase character: [A-Z]
\t matches <Tab>
\r matches <CR>
\n matches an end-of-line
\1 Matches the same string that was matched by the first sub-expression in \( and \). Example: "\([a-z]\).\1" matches "ata", "ehe", "tot", etc.

关于VIM中的[](:h /collection)


在其他语言中perl/javascript/ruby中,方括号可以使用转义字符代替的字符集,如"\s", "\w"等,在VIM中只能在方括号中使用字符序列,也可以像其他编程语言一样放入"a-z", "0-9"等,但不能用转义字符,这点在使用时比较不顺手.

以上是个人认为比较常用并且较为简单的部分内容,更多神奇的VIM正则查找,以及结合一些VIM内置方法进行正则替换,则需要仔细阅读帮助手册说明.要熟练掌握VIM,就需要多查手册,多实践.

Friday, October 08, 2010

个人一直使用的一套ubuntu主题

http://www.bisigi-project.org/?page_id=8&lang=en上下载安装,在ubuntu上用apt-get安装很简单:

$>
sudo add-apt-repository ppa:bisigi/ppa && sudo apt-get update
$> sudo aptitude install showtime-theme

其他更多主题可以选择,都很不错.

ubuntu 10.10 自带的Ambiance配合mac4Lin的emerald主题也很酷.

Friday, October 01, 2010

vim中匹配换行符的正则表达式

\s* 匹配0或多个空白(比如空格,Tab等,不匹配换行)
VIM里面,如果要连换行一起匹配,则加个下划线,比如\_s匹配包括换行在内的空白,而\_.匹配包括换行在内的任意字符 (注意,后面有个小数点)

References:
VIM正则表达式查找替换

Ubuntu下Nvidia显卡不定时闪屏的解决

解决方法:

在配置文件中添加一行:
$> sudo vi /etc/modprobe.d/nvidia-kernel-nkc
options nvidia NVreg_Mobile=1 NVreg_RegistryDwords="PerfLevelSrc=0x2222"

更多信息可查看ubuntu的bug list:Occasional screen-wide "blink" when using opengl apps (compiz also) and Nvidia cards

References:
[分享]解决Nvidia显卡不定时闪屏问题

Friday, September 17, 2010

maven 学习笔记

maven的POM文件有一份默认的Super POM,所有新建的POM文件都是继承自此POM,这个Super POM的build配置部分如下:


<build>
<directory>target</directory>
<outputDirectory>target/classes</outputDirectory>
<finalName>${artifactId}-${version}</finalName>
<testOutputDirectory>target/test-classes</testOutputDirectory>
<sourceDirectory>src/main/java</sourceDirectory>
<scriptSourceDirectory>src/main/scripts</scriptSourceDirectory>
<testSourceDirectory>src/test/java</testSourceDirectory>
<resources>
<resource>
<directory>src/main/resources</directory>
</resource>
</resources>
<testResources>
<testResource>
<directory>src/test/resources</directory>
</testResource>
</testResources>
</build>

在WEB项目中会希望在mvn package之后将maven导入的lib文件自动复制到wepapp/WEB-INF/lib下方便测试,如果maven导入的依赖文件发生变化时,将lib目录下的文件先清理之后,再重新加入,同时也可以将其他的lib目录的文件一起复制到webapp/WEB-INF/lib下,要达到这个功能,可以用maven去运行ant的可执行脚本,maven-antrun-plugin 就达到这个目的,只要将ant的任务放到tasks标签里,格式如:

<project>
<build>
<plugins>
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<version>1.4</version>
<executions>
<execution>
<phase> <!-- a lifecycle phase --> </phase>
<configuration>
<tasks>

<!--
Place any Ant task here. You can add anything
you can add between <target> and </target> in a
build.xml.
-->

</tasks>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>

在linux中,因为默认使用了openjdk,所以在使用maven时,会遇到提示 tools.jar 包的依赖问题,可以在POM中添加下面的配置,引入 tools.jar:

<profiles>
<profile>
<id>default-tools.jar</id>
<activation>
<property>
<name>java.vendor</name>
<value>Sun Microsystems Inc.</value>
</property>
</activation>
<dependencies>
<dependency>
<groupId>com.sun</groupId>
<artifactId>tools</artifactId>
<version>1.4.2</version>
<scope>system</scope>
<systemPath>${java.home}/../lib/tools.jar</systemPath>
</dependency>
</dependencies>
</profile>
</profiles>

在项目目录下运行mvn命令时,如果看到 jvm 1.3 的编译错误,则需要为maven指定源码和JVM的版本,如下:

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.0.2</version>
<configuration>
<source>1.5</source>
<target>1.5</target>
</configuration>
</plugin>
</plugins>
</build>

在项目下面运行 jUnit test 时,如果运行 mvn test 是成功的,但是单独在一个test文件是运行jUnit test时则报错说 Class not found(或者二者相反),这个是因为maven默认为test的java文件生成的class放到target/test-classes,项目本身的classes output目录可能是设置在webapp/WEB-INF/classes目录中的,二者不一致才造成这个问题,需要在build标签下面设置 outputDirectory 和 testOutputDirectory 这二个标签,让maven也到webapp/WEB-INF/classes目录中查找编译后的文件。

maven中常用命令说明,项目打包和安装到仓库中,在项目下运行:
$> mvn package
为当前项目生成一个jar文件或者是war文件,这个由POM文件配置中的packaging参数决定是jar还是war,运行成功后,会在${basedir}/target目录下生成文件。如果需要将生成的jar文件放到maven仓库中(默认是: ~/.m2/repository),则运行命令:
$> mvn install

为eclipse IDE生成maven项目,配合eclipse maven插件如: m2e - http://m2eclipse.sonatype.org/sites/m2e 一起使用:
$> mvn eclipse:eclipse

运行当前项目中的jUnit test,maven是使用maven-surefire-plugin完成test的,关于这个插件,可以查看 http://tianya23.blog.51cto.com/1081650/386012 中的说明,可以忽略或者指定测试文件的名字,以及在test发生错误如何不影响打包:
$> mvn test

根据上面的maven知识点,配置出符合项目需求的POM配置的build部分如下:

<build>
<sourceDirectory>src/main/java</sourceDirectory>
<resources>
<resource>
<directory>src/main/resources</directory>
</resource>
</resources>

<outputDirectory>src/main/webapp/WEB-INF/classes</outputDirectory>
<testOutputDirectory>src/main/webapp/WEB-INF/classes</testOutputDirectory>

<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.6</source>
<target>1.6</target>
<encoding>UTF-8</encoding>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<configuration>
<webXml>src/main/webapp/WEB-INF/web.xml</webXml>
</configuration>
</plugin>

<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<id>copy-lib-src-webapp</id>
<phase>install</phase>
<configuration>
<tasks>
<delete dir="src/main/webapp/WEB-INF/lib" />
<copy todir="src/main/webapp/WEB-INF/lib">
<fileset dir="target/package-name-0.0.1-SNAPSHOT/WEB-INF/lib">
<include name="*" />
</fileset>
</copy>
<copy todir="src/main/webapp/WEB-INF/lib">
<fileset dir="lib">
<include name="*" />
</fileset>
</copy>
</tasks>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>

</build>

可以从下面的maven仓库中查找包的依赖和maven配置:
* http://repository.apache.org
* http://www.artifact-repository.org
* http://mvnrepository.com
* http://www.mvnbrowser.com
* http://www.jarvana.com
* http://mavensearch.net

Reference:
http://maven.apache.org/general.html
http://maven.apache.org/plugins/
http://maven.apache.org/pom.html
http://maven.apache.org/settings.html
http://maven.apache.org/guides/getting-started/index.html
http://maven.apache.org/plugins/maven-antrun-plugin/usage.html
http://maven.apache.org/plugins/maven-surefire-plugin/usage.html
http://maven.apache.org/guides/introduction/introduction-to-the-pom.html

Monday, August 30, 2010

maven:install在命令行构建成功而m2eclipse中构建失败

在命令行中用
$> mvn package
能构建成功,但在eclipse中用m2eclipse插件则构建失败,提示信息摘录部分如下:
[ERROR] FATAL ERROR
[INFO] ------------------------------------------------------------------------
[INFO] dependenciesInfo : dependenciesInfo
---- Debugging information ----
message : dependenciesInfo : dependenciesInfo
cause-exception : com.thoughtworks.xstream.mapper.CannotResolveClassException
cause-message : dependenciesInfo : dependenciesInfo
class : org.apache.maven.plugin.war.util.WebappStructure
required-type : org.apache.maven.plugin.war.util.WebappStructure
path : /webapp-structure/dependenciesInfo

在网上查了一些资料,主要参考: http://jira.codehaus.org/browse/MWAR-187
之所以命令行能成功,而插件不成功,主要原因是二者的maven版本不一样造成的,因为2eclipse使用embedded Maven 3 (Maven Embedder 3.0.-SNAPSHOT),可以通过window->perferences->Maven->Installations查看到。

解决这个问题的方法在上文链接中也有提到:
1、Maven版本降级
2、最快捷的方式是禁用缓存(webapp-cache.xml)

<configuration>
<useCache>false</useCache>
</configuration>

3、在eclipse中用m2eclipse插,先运行mvn clean后(这样会删除target目录),再运行mvn install

References: http://jira.codehaus.org/browse/MWAR-187

Sunday, August 08, 2010

install sun java on ubuntu 10.04

Sun Java moved to the Partner repository

For Ubuntu 10.04 LTS, the sun-java6 packages have been dropped from the Multiverse section of the Ubuntu archive. It is recommended that you use openjdk-6 instead.

If you can not switch from the proprietary Sun JDK/JRE to OpenJDK, you can install sun-java6 packages from the Canonical Partner Repository. You can configure your system to use this repository via command-line:

$> add-apt-repository "deb http://archive.canonical.com/ lucid partner"
---------------
and then:
$> sudo add-apt-repository "deb http://archive.canonical.com/ lucid partner"
$> sudo apt-get update
$> sudo apt-get install sun-java6-jdk

Saturday, July 31, 2010

find mysqld configure file position

检查mysqld读取配置文件的位置,可以用下面这个命令:
$> mysqld --verbose --help|grep -A 1 'Default options'

摘自《高性能MySQL》

Tuesday, July 20, 2010

google chrome about command list

about:memory
about:stats
about:network
about:internets
about:histograms
about:dns
about:cache
about:crash
about:plugins
about:version

ubuntu 9.10 上安装facebook XHP

http://wiki.github.com/facebook/xhp/building-xhp

1. Download xhp from http://github.com/facebook/xhp/downloads

$> sudo apt-get install php gcc flex bison re2c

$> phpize
$> ./configure
$> make
$> sudo make install

3. add xhp config file to php
$> sudo vi /etc/php5/conf.d/xhp.ini
# add below line to xhp.ini file
extension=xhp.so

4. test xhp, if you get a syntax error then XHP is not working.
$> php -r 'echo "XHP!\n"; exit; <a />;'

5. xhp configure
xhp.idx_expr = {1 | 0} (default: 0): This is a bonus feature of XHP which adds support for the [] operator on the return value of a function. If you enable this you will be able to write code like foo()['bar'] without hitting syntax errors. This configuration is mainly around to fix a personal annoyance with PHP's grammar, and has nothing to do with XHP's core features.

xhp.include_debug = {1 | 0} (default: 1): You can disable XHP node annotations by turning this setting off. This may in some cases marginally increase performance of XHP-heavy code. It is highly recommended that you leave this on, as tracking errors in XHP applications is very difficult without annotations.

Saturday, July 17, 2010

acme网站上几个http小工具

1. http_load: http://www.acme.com/software/http_load/

同apache ab和JMeter一样可以对数据库和web服务器作压力测试。
$> vi urls


$> ./http_load -rate 5 -seconds 10 urls
49 fetches
2 max parallel
289884 bytes
in 10.0148 seconds 5916 mean bytes/connection 4.89274 fetches/sec
28945.5 bytes/sec msecs/connect: 28.8932 mean
44.243 max
24.488 min msecs/first-response: 63.5362 mean
81.624 max
57.803 min
HTTP response codes: code 200 -- 49

2. http_ping: http://www.acme.com/software/http_ping/

与ping命令类似,但是是通过http请求来检查服务器,而非ICMP请求服务器返回响应。

$> http_ping http://www.example.com/
7816 bytes from http://www.example.com/:
246.602 ms (9.923c/23.074r/213.605d)
7816 bytes from http://www.example.com/:
189.997 ms (11.619c/22.971r/155.407d)
7816 bytes from http://www.example.com/:
190.463 ms (8.994c/25.091r/156.378d)
7816 bytes from http://www.example.com/:
190.07 ms (9.234c/23.9r/156.936d)
7816 bytes from http://www.example.com/:
190.706 ms (10.142c/46.579r/133.985d)
--- http://www.example.com/
http_ping statistics
--- 5 fetches started, 5 completed (100%), 0 failures (0%), 0 timeouts (0%)
total min/avg/max = 189.997/201.568/246.602 ms
connect min/avg/max = 8.994/9.9824/11.619 ms
response min/avg/max = 22.971/28.323/46.579 ms
data min/avg/max = 133.985/163.262/213.605 ms

Tuesday, July 06, 2010

mongodb tutorial and references

Introduction


MongoDB is a collection-oriented, schema-free document database.

By collection-oriented, we mean that data is grouped into sets that are called 'collections'. Each collection has a unique name in the database, and can contain an unlimited number of documents. Collections are analogous to tables in a RDBMS, except that they don't have any defined schema.

By schema-free, we mean that the database doesn't need to know anything about the structure of the documents that you store in a collection. In fact, you can store documents with different structure in the same collection if you so choose.

By document, we mean that we store data that is a structured collection of key-value pairs, where keys are strings, and values are any of a rich set of data types, including arrays and documents. We call this data format "BSON" for "Binary Serialized dOcument Notation."

MongoDB is a server process that runs on Linux, Windows and OS X. It can be run both as a 32 or 64-bit application. We recommend running in 64-bit mode, since Mongo is limited to a total data size of about 2GB for all databases in 32-bit mode.

The MongoDB process listens on port 27017 by default (note that this can be set at start time - please see Command Line Parameters for more information).

MongoDB stores its data in files (default location is /data/db/), and uses memory mapped files for data management for efficiency.

Example data of mongodb


$> bin/mongo --shell slides.js


/*
* Slides for presentation on Mastering the MongoDB Shell.
* Copyright 2010 Mike Dirolf (http://dirolf.com)
* This work is licensed under the Creative Commons
* Attribution-Noncommercial-Share Alike 3.0 United States License. To
* view a copy of this license, visit
* http://creativecommons.org/licenses/by-nc-sa/3.0/us/ or send a
* letter to Creative Commons, 171 Second Street, Suite 300, San
* Francisco, California, 94105, USA.
* Originally given at MongoSF on 4/30/2010.
* Modified and given at MongoNYC on 5/21/2010.
* To use: run `mongo --shell slides.js`
*/
db = db.getSisterDB("shell");
db.dropDatabase();
// some sample data
for (var i = 0; i < 1000; i += 1) {
db.data.save({
x: i
});
} // "slides"
db.deck.save({
slide: 0,
welcome: "to MongoNYC!",
hashtag: "#mongonyc",
mirror: "http://confmirror.10gen.com/"
});
db.deck.save({
slide: 1,
title: "Mastering the MongoDB Shell",
who: "Mike Dirolf, 10gen",
handle: "@mdirolf"
});
db.deck.save({
slide: 2,
question: "what is the shell?",
answer: "a better Powerpoint?"
});
db.deck.save({
slide: 3,
question: "what is the shell?",
answer: "a full JavaScript environment"
});
db.deck.save({
slide: 4,
question: "what is the shell?",
answer: "a reference MongoDB client"
});
db.deck.save({
slide: 5,
"use cases": ["administrative scripting", "exploring and debugging", "learning (and teaching!)"]
});
db.deck.save({
slide: 6,
repl: ["arrows for history", "^L"]
});
db.deck.save({
slide: 7,
"getting help": ["help", "db.help", "db.foo.help"]
});
db.deck.save({
slide: 8,
"show": ["dbs", "collections", "users", "profile"]
});
db.deck.save({
slide: 9,
navigating: "databases",
how: "'use' or 'db.getSisterDB'"
});
db.deck.save({
slide: 10,
navigating: "collections",
how: "dots, brackets, or 'db.getCollection'",
note: "careful with names like foo-bar"
});
db.deck.save({
slide: 11,
"basic operations": ["insert", "findOne", "find", "remove"]
});
db.deck.save({
slide: 12,
"fun with cursors": ["auto-iteration", "it"]
});
db.deck.save({
slide: 13,
"error checking": "auto 'db.getLastError'"
});
db.deck.save({
slide: 14,
"commands": ["count", "stats", "repairDatabase"],
"meta": "listCommands"
});
db.deck.save({
slide: 15,
"pro tip!": "viewing JS source"
});
db.deck.save({
slide: 16,
"getting help": "--help"
});
db.deck.save({
slide: 17,
scripting: "run .js files",
tools: ["--eval", "--shell", "runProgram"]
});
db.deck.save({
slide: 18,
warning: "dates in JS suck"
});
db.deck.save({
slide: 19,
warning: "array iteration in JS sucks"
});
db.deck.save({
slide: 20,
warning: "numeric types in JS suck"
});
db.deck.save({
slide: 21,
so: "why JS?"
});
db.deck.save({
slide: 22,
homework: ["convince 2 friends to try MongoDB", "send feedback @mdirolf"]
});
db.deck.save({
slide: 23,
url: "github.com/mdirolf/shell_presentation",
questions: "?"
}); // current slide
var current = 0; // print current slide and advance
var next = function() {
var slide = db.deck.findOne({
slide: current
});
if (slide) {
current++;
delete slide._id;
delete slide.slide;
print(tojson(slide, null, false));
} else {
print("The End!");
}
}; // go to slide and print
var go = function(n) {
current = n;
next();
}; // repeat the previous slide
var again = function() {
current--;
next();
};


Basic Commands of mongodb



http://www.mongodb.org/display/DOCS/List+of+Database+Commands
http://rezmuh.sixceedinc.com/2010/02/basic-commands-to-get-you-started-with-mongodb.html

Privileged Commands

Certain operations are for the database administrator only. These privileged operations may only be performed on the special database named admin.

Start the server like this:
$> bin/mongod --dbpath /path/to/data

Stop it with Ctrl-C or with kill (but don’t use kill -9, which doesn’t give the server a chance to shut down cleanly and flush data to disk).

Viewing stats without the mongo console:
Visit http://localhost:28017 (28017 is the port the server is running on, plus 1000) to get an overview of what’s going on.
You can also query http://localhost:28017/_status to see the same data that db.serverStatus() returns, but in JSON format.

Use client command console:

$> bin/mongo

> use admin;
> db.addUser('dummy, 'dummy') // create new user with password "dummy"
> db.runCommand("shutdown"); // shut down the database

$> bin/mongo -u dummy -p dummy admin

Create a new Database:

> use new_database
> db.addUser('dummy', 'differentpassword')

$ bin/mongo -u dummy -p differentpassword new_database


Deleting a database:

> use [db name]
> db.dropDatabase()

Working with Collections (Tables):
> show collections

Lists all the available databases:
> show dbs

To get a list of data in a specific collection, use:
> db.[collection name].find()

Directory Global functions and properties:

> for(var o in this) {
print(o);
}
__quiet
chatty
friendlyEqual
doassert
assert
argumentsToArray
isString
isNumber
isObject
tojson
tojsonObject
shellPrint
printjson
shellPrintHelper
shellHelper
help
Random
killWithUris
Geo
connect
MR
MapReduceResult
__lastres__
sleep_
sleep
quit_
quit
getMemInfo_
getMemInfo
_srand_
_srand
_rand_
_rand
_isWindows_
_isWindows
_startMongoProgram_
_startMongoProgram
runProgram_
runProgram
runMongoProgram_
runMongoProgram
stopMongod_
stopMongod
stopMongoProgram_
stopMongoProgram
stopMongoProgramByPid_
stopMongoProgramByPid
rawMongoProgramOutput_
rawMongoProgramOutput
clearRawMongoProgramOutput_
clearRawMongoProgramOutput
removeFile_
removeFile
listFiles_
listFiles
resetDbpath_
resetDbpath
copyDbpath_
copyDbpath
_parsePath
_parsePort
createMongoArgs
startMongodTest
startMongod
startMongodNoReset
startMongos
startMongoProgram
startMongoProgramNoConnect
myPort
ShardingTest
printShardingStatus
MongodRunner
ReplPair
ToolTest
ReplTest
allocatePorts
SyncCCTest
db
hex_md5_
hex_md5
version_
version
i
current
next
__iscmd__
___it___
it

Directory properties of db:

> for(var o in db){
print(o);
}
_mongo
_name
shellPrint
getMongo
getSisterDB
getName
stats
getCollection
commandHelp
runCommand
_dbCommand
_adminCommand
addUser
removeUser
__pwHash
auth
createCollection
getProfilingLevel
dropDatabase
shutdownServer
cloneDatabase
cloneCollection
copyDatabase
repairDatabase
help
printCollectionStats
setProfilingLevel
eval
dbEval
groupeval
groupcmd
group
_groupFixParms
resetError
forceError
getLastError
getLastErrorObj
getLastErrorCmd
getPrevError
getCollectionNames
tojson
toString
currentOp
currentOP
killOp
killOP
getReplicationInfo
printReplicationInfo
printSlaveReplicationInfo
serverBuildInfo
serverStatus
version
printShardingStatus

> db.shell.help()
DBCollection help
db.foo.count()
db.foo.dataSize()
db.foo.distinct( key ) - eg. db.foo.distinct( 'x' )
db.foo.drop() drop the collection
db.foo.dropIndex(name)
db.foo.dropIndexes()
db.foo.ensureIndex(keypattern,options) - options should be an object with these possible fields: name, unique, dropDups
db.foo.reIndex()
db.foo.find( [query] , [fields]) - first parameter is an optional query filter. second parameter is optional set of fields to return.
e.g. db.foo.find( { x : 77 } , { name : 1 , x : 1 } )
db.foo.find(...).count()
db.foo.find(...).limit(n)
db.foo.find(...).skip(n)
db.foo.find(...).sort(...)
db.foo.findOne([query])
db.foo.findAndModify( { update : ... , remove : bool [, query: {}, sort: {}, 'new': false] } )
db.foo.getDB() get DB object associated with collection
db.foo.getIndexes()
db.foo.group( { key : ..., initial: ..., reduce : ...[, cond: ...] } )
db.foo.mapReduce( mapFunction , reduceFunction , <optional params> )
db.foo.remove(query)
db.foo.renameCollection( newName , <droptarget> ) renames the collection.
db.foo.runCommand( name , <options> ) runs a db command with the given name where the 1st param is the colleciton name
db.foo.save(obj)
db.foo.stats()
db.foo.storageSize() - includes free space allocated to this collection
db.foo.totalIndexSize() - size in bytes of all the indexes
db.foo.totalSize() - storage allocated for all data and indexes
db.foo.update(query, object[, upsert_bool, multi_bool])
db.foo.validate() - SLOW
db.foo.getShardVersion() - only for use with sharding

> db.help()
DB methods:
db.addUser(username, password[, readOnly=false])
db.auth(username, password)
db.cloneDatabase(fromhost)
db.commandHelp(name) returns the help for the command
db.copyDatabase(fromdb, todb, fromhost)
db.createCollection(name, { size : ..., capped : ..., max : ... } )
db.currentOp() displays the current operation in the db
db.dropDatabase()
db.eval(func, args) run code server-side
db.getCollection(cname) same as db['cname'] or db.cname
db.getCollectionNames()
db.getLastError() - just returns the err msg string
db.getLastErrorObj() - return full status object
db.getMongo() get the server connection object
db.getMongo().setSlaveOk() allow this connection to read from the nonmaster member of a replica pair
db.getName()
db.getPrevError()
db.getProfilingLevel()
db.getReplicationInfo()
db.getSisterDB(name) get the db at the same server as this onew
db.killOp(opid) kills the current operation in the db
db.printCollectionStats()
db.printReplicationInfo()
db.printSlaveReplicationInfo()
db.printShardingStatus()
db.removeUser(username)
db.repairDatabase()
db.resetError()
db.runCommand(cmdObj) run a database command. if cmdObj is a string, turns it into { cmdObj : 1 }
db.serverStatus()
db.setProfilingLevel(level,) 0=off 1=slow 2=all
db.shutdownServer()
db.stats()
db.version() current version of the server

> help()
HELP
show dbs show database names
show collections show collections in current database
show users show users in current database
show profile show most recent system.profile entries with time >= 1ms
use <db name> set curent database to <db name>
db.help() help on DB methods
db.foo.help() help on collection methods
db.foo.find() list objects in collection foo
db.foo.find( { a : 1 } ) list objects in foo where a == 1
it result of the last line evaluated; use to further iterate

If I insert the same Document twice, it does not raise an Error?



Using PyMongo for example, why does inserting the same document (read same _id) more than once not raise an error? When we need to detect if a document already exists in the database, we could try catching DuplicateKeyError on insert. Below we explicitly insert the document with the same _id twice but the exception is never raised. Why?

try:
doc = { _id: '123123' }
db.foo.insert(doc)
db.foo.insert(doc)
except pymongo.errors.DuplicateKeyError, error:
print("Same _id inserted twice:", error)

The answer is that DuplicateKeyError will only be raised if we do the insert in safe mode i.e. db.foo.insert(doc, safe=True). The reason why we do not see an error raised with most drivers but with MongoDB's interactive shell is because the shell does a safe insert by default whereas, with most drivers, it is the developers choice whether or not to use a safe insert.

MongoDB's interactive shell:
> doc = {_id: 123421, name: 'test'};
{ "_id" : 123421, "name" : "test" }
> db.deck.insert(doc)
> db.deck.insert(doc)
E11000 duplicate key error index: shell.deck.$_id_ dup key: { : 123421.0 }

Backup



http://effectif.com/mongodb/mongo-administration

There are basically two approaches to backing up a Mongo database:

1. mongodump and mongorestore are the classic approach. Dumps the contents of the database to files. The backup is stored in the same format as Mongo uses internally, so is very efficient. But it’s not a point-in-time snapshot.
2. To get a point-in-time snapshot, shut the database down, copy the disk files (e.g. with cp) and then start mongod up again.

Alternatively, rather than shutting mongod down before making your point-in-time snapshot, you could just stop it from accepting writes:

> db._adminCommand({fsync: 1, lock: 1})
{
"info" : "now locked against writes, use db.$cmd.sys.unlock.findOne() to unlock",
"ok" : 1
}

To unlock the database again, you need to switch to the admin database and then unlock it:

> use admin
switched to db admin
> db.$cmd.sys.unlock.findOne()
{ "ok" : 1, "info" : "unlock requested" }


If you don’t switch to the admin database first you’ll get an unauthorized error:

> db._adminCommand({fsync: 1, lock: 1})
{
"info" : "now locked against writes, use db.$cmd.sys.unlock.findOne() to unlock",
"ok" : 1
}

> db.$cmd.sys.unlock.findOne()
{ "err" : "unauthorized" }


You can take a point in time snapshot from a slave just as easily as your master database, which avoids downtime. This is one of the reasons that running a slave is so strongly recommended…

What RAID should I use?



http://www.mongodb.org/display/DOCS/Developer+FAQ

We recommend not using RAID-5, but rather, RAID-10 or the like. Both will work of course.

Replication



Do it (did you read the previous section?). Seriously.

Start your master and slave up like this:

$ mongod --master --oplogSize 500
$ mongod --slave --source localhost:27017 --port 3000 --dbpath /data/slave

When seeding a new slave server from master use the --fastsync option.

You can see what’s going on with these two commands:

> db.printReplicationInfo() # tells you how long your oplog will last
> db.printSlaveReplicationInfo() # tells you how far behind the slave is

If the slave isn’t keeping up, how do you find out what’s going on? Check the mongo log for any recent errors. Try connecting with the mongo console. Try running queries from the console to see if everything is working. Run the status commands above to try and find out which database is taking up resources. If you can’t work it out hop on the IRC channel; Mathias says they’ll be very responsive.

What is the difference between MongoDB and RDBMSs



http://sunoano.name/ws/mongodb.html#faqs

MySQL, PostgreSQL, ...
----------------------
Server:Port
- Database
- Table
- Row

MongoDB
--------
Server:Port
- Database
- Collection
- Document

The concept of server and database are very similar. But the concept of table and collection are quite different. In RDBMSs a table is a rectangle. It is all columns and rows. Each row has a fixed number of columns, if we add a new column, we add that column to each and every row.

In MongoDB a collection is more like a really big box and each document is like a little bag of stuff in that box. Each bag contains whatever it needs in a totally flexible manner (read schema-less). However, schema-less does not equal type-less i.e. it is just that any document has its own schema, which it may or may not share with any other document. In practice it is normal to have the same schema for all the documents in collection.

Clone Database



http://www.mongodb.org/display/DOCS/Clone+Database

MongoDB includes commands for copying a database from one server to another.

// copy an entire database from one name on one server to another
// name on another server. omit <from_hostname> to copy from one
// name to another on the same server.
db.copyDatabase(<from_dbname>, <to_dbname>, <from_hostname>);
// if you must authenticate with the source database
db.copyDatabase(<from_dbname>, <to_dbname>, <from_hostname>, <username>, <password>);
// in "command" syntax (runnable from any driver):
db.runCommand( { copydb : 1, fromdb : ..., todb : ..., fromhost : ... } );
// command syntax for authenticating with the source:
n = db.runCommand( { copydbgetnonce : 1, fromhost: ... } );
db.runCommand( { copydb : 1, fromhost: ..., fromdb: ..., todb: ..., username: ..., nonce: n.nonce, key: <hash of username, nonce, password > } );

// clone the current database (implied by 'db') from another host
var fromhost = ...;
print("about to get a copy of database " + db + " from " + fromhost);
db.cloneDatabase(fromhost);
// in "command" syntax (runnable from any driver):
db.runCommand( { clone : fromhost } );

What might be a use case for --fork?



The --fork switch forks a mongod process of another exiting process. One use case where this is pretty handy for example is if we SSH into a remote machine and start a MongoDB and leave right away after a new mongod got started.

The fastest way to do so would simply be to append /path/to/mongod --fork to the SSH command line since this will skip creating an intermediate shell and bring up a remote mongod right away.

Are there any Reasons not to use MongoDB?



http://sunoano.name/ws/mongodb.html#faqs

1. We need transactions (read ACID).
2. Our data is very relational.
3. Related to 2, we want to be able to do joins on the server (but can not do embedded objects / arrays).
4. We need triggers on our tables. There might be triggers available soon however.
5. Related to 4, we rely on triggers (or similar functionality) to do cascading updates. or deletes. As for #4, this issue probably goes away once triggers are available.
6. We need the database to enforce referential integrity (MongoDB has no notion of this at all).
7. If we need 100% per node durability.

Use Cases



http://www.mongodb.org/display/DOCS/Comparing+Mongo+DB+and+Couch+DB

It may be helpful to look at some particular problems and consider how we could solve them.

* if we were building Lotus Notes, we would use Couch as its programmer versioning reconciliation/MVCC model fits perfectly. Any problem where data is offline for hours then back online would fit this. In general, if we need several eventually consistent master-master replica databases, geographically distributed, often offline, we would use Couch.
* if we had very high performance requirements we would use Mongo. For example, web site user profile object storage and caching of data from other sources.
* if we were building a system with very critical transactions, such as bond trading, we would not use MongoDB for those transactions -- although we might in hybrid for other data elements of the system. For something like this we would likely choose a traditional RDBMS.
* for a problem with very high update rates, we would use Mongo as it is good at that. For example, updating real time analytics counters for a web sites (pages views, visits, etc.)

Generally, we find MongoDB to be a very good fit for building web infrastructure.

ruby driver of mongodb tutorial



http://api.mongodb.org/ruby/1.0.3/index.html
http://www.mongodb.org/display/DOCS/Ruby+Tutorial
http://github.com/mongodb/mongo-ruby-driver

$> gem update --system
$> gem install mongo
$> gem install bson
$> gem install bson_ext

Making a Connection:
# db = Mongo::Connection.new.db("shell")
# db = Mongo::Connection.new("localhost").db("shell")
db = Mongo::Connection.new("localhost", 27017).db("shell")


Listing All Databases:
m = Mongo::Connection.new # (optional host/port args)
m.database_names.each { |name| puts name }
m.database_info.each { |info| puts info.inspect}


Dropping a Database:
m.drop_database('database_name')


Authentication (Optional):
auth = db.authenticate(my_user_name, my_password)


Getting a Collection:
deck = db.collection("deck")


Inserting a Document:
deck = db['deck']

doc = {"name" => "MongoDB", "type" => "database", "count" => 1,
"info" => {"x" => 203, "y" => '102'}}
deck.insert(doc)
100.times { |i| deck.insert("i" => i) }


Finding the First Document In a Collection using find_one():
my_doc = deck.find_one()
puts my_doc.inspect


Note the _id element has been added automatically by MongoDB to your document. Remember, MongoDB reserves element names that start with _ for internal use.

Counting Documents in a Collection:
puts deck.count()


Using a Cursor to get all of the Documents:
deck.find.each { |row| puts row.inspect }


Getting a Single Document with a Query:
deck.find("i" => 71).each { |row| puts row.inspect }


Getting a Set of Documents With a Query:
deck.find("i" => {"$gt" => 50}).each { |row| puts row }


Querying with Regular Expressions:
deck.find("question" => /w/i).each { |row| puts row }


Creating An Index:
To create an index, you specify an index name and an array of field names to be indexed, or a single field name.
deck.create_index("i")
# deck.create_index([["i", Mongo::ASCENDING]])


Getting a List of Indexes on a Collection:
deck.index_information


Ruby driver example


require 'rubygems' not necessary for Ruby 1.9
require 'mongo'
# 需要先导入slides.js之后,再进行下面的ruby代码测试
db = Mongo::Connection.new.db("mydb")
db = Mongo::Connection.new("localhost").db("mydb")
m = Mongo::Connection.new("localhost", 27017)
db = m.db("shell")
puts db.inspect
m.database_names.each { |name| puts name }
m.database_info.each { |info| puts info.inspect}
db.collection_names.each { |name| puts name }
deck = db['deck']
doc = {"name" => "MongoDB", "type" => "database", "count" => 1,
"info" => {"x" => 203, "y" => '102'}}
deck.insert(doc)
100.times { |i| deck.insert("i" => i) }
my_doc = deck.find_one()
puts my_doc.inspect
puts deck.count()
deck.find.each { |row| puts row.inspect }
deck.find("i" => 71).each { |row| puts row.inspect }
deck.find("i" => {"$gt" => 90}).each { |row| puts row }
deck.find("i" => {"$gt" => 20, "$lte" => 30}).each { |row| puts row }
deck.find("question" => /w/i).each { |row| puts row }
deck.create_index([["i", Mongo::ASCENDING]])
p deck.index_information
db.validate_collection('deck')


References: http://www.mongodb.org/display/DOCS/Developer+Zone

Friday, July 02, 2010

Installation XEN 4.0 from source

以下指令使用root用户执行


$> sudo su -

$> apt-get install bcc bin86 gawk bridge-utils iproute libcurl3 libcurl4-openssl-dev bzip2 module-init-tools transfig tgif texinfo texlive-latex-base texlive-latex-recommended texlive-fonts-extra texlive-fonts-recommended pciutils-dev mercurial build-essential make gcc libc6-dev zlib1g-dev python python-dev python-twisted libncurses5-dev patch libvncserver-dev libsdl-dev libjpeg62-dev iasl libbz2-dev e2fslibs-dev git-core uuid-dev

$> hg clone http://xenbits.xen.org/xen-4.0-testing.hg
$> cd xen-4.0-testing.hg

这个指令会下载linux-2.6-pvops.git,需要较长时间
$> make world


这个指令会安装xen.gz和vmlinuz-2.6.31到系统/boot目录下,vmlinuz-2.6.31这个linux kernel文件在下面会重新设置参数并生成,可查看后面linux kernel编译步骤详细说明。
$> make install


不修改.config文件,可以直接重新生成xen, kernels, tools, docs到dist目录中
$> make dist


修改了.config文件,可以用以下指令重新生成kernel
$> make linux-2.6-xen-config CONFIGMODE=menuconfig     (or xconfig)
$> make linux-2.6-xen-build
$> make linux-2.6-xen-install

linux kernel编译步骤

$> git clone git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git linux-2.6-xen

这个指令会从xen.git仓库中下载props dom0 kernel 2.6.31.x,大概是1.6G左右,需要不少时间
$> cd linux-2.6-xen

这个指令的作用是检查源码是否有.o文件和依赖问题,从全新的源码编译时不需要这一步。make mrproper主要清除环境变量及配置文件
$ make mrproper


设置linux kernel编译参数,在图形界面中输入斜杠"/"可以查询"XEN_DOM0"、"PAE"、"HIGHPTE"等参数设置
$> make menuconfig


退出并保存配置文件之后,手工调整.config文件,并且在内核参数设置中需要注意:
1、如果编译32位的内核,XEN需要有PAE支持,(Processor type and features -> High Memory Support (64GB) -> PAE (Physical Address Extension) Support),对于64位内核PAE不需要,32位的操作系统最大内存支持不到4G,安装了PAE之后可以支持64G。
2、编译32位的内核,必须设置CONFIG_HIGHPTE=n。
3、要为DOM0开启ACPI功能。
4、在.config文件中添加以下选项,并重新运行make menuconfig,并检查XEN_DOM0等参数设置状态

CONFIG_ACPI_PROCFS=y
CONFIG_XEN=y
CONFIG_XEN_MAX_DOMAIN_MEMORY=32
CONFIG_XEN_SAVE_RESTORE=y
CONFIG_XEN_DOM0=y
CONFIG_XEN_PRIVILEGED_GUEST=y
CONFIG_XEN_PCI=y
CONFIG_PCI_XEN=y
CONFIG_XEN_BLKDEV_FRONTEND=m
CONFIG_NETXEN_NIC=m
CONFIG_XEN_NETDEV_FRONTEND=m
CONFIG_XEN_KBDDEV_FRONTEND=m
CONFIG_HVC_XEN=y
CONFIG_XEN_FBDEV_FRONTEND=m
CONFIG_XEN_BALLOON=y
CONFIG_XEN_SCRUB_PAGES=y
CONFIG_XEN_DEV_EVTCHN=y
CONFIG_XEN_BACKEND=y
CONFIG_XEN_BLKDEV_BACKEND=y
CONFIG_XEN_NETDEV_BACKEND=y
CONFIG_XENFS=y
CONFIG_XEN_COMPAT_XENFS=y
CONFIG_XEN_XENBUS_FRONTEND=m
CONFIG_XEN_PCIDEV_FRONTEND=y

在RHEL5和CentOS5中,需要在.config文件中加入以下2个参数:

CONFIG_SYSFS_DEPRECATED=y
CONFIG_SYSFS_DEPRECATED_V2=y


要使用initrd,编译内核时必须选择以下两项:

CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_INITRD=y

重新运行menuconfig,检查配置,最后保存配置文件
$> make menuconfig
$> make install

这个指令会将内核映象和相应的System.map拷贝到/boot目录下。

可以查看指令运行完成后,在/boot目录下新增加的文件列表
$> ll --sort=time /boot/
total 214594
-rw-r--r-- 1 root root 109K 2010-07-02 11:51 config-2.6.31.13
-rw-r--r-- 1 root root 1.6M 2010-07-02 11:51 System.map-2.6.31.13
-rw-r--r-- 1 root root 3.9M 2010-07-02 11:51 vmlinuz-2.6.31.13
...


生成初始化镜像文件(initialed ramdisk)
$> cd /boot
$> depmod 2.6.31.13
# ubuntu中生成初如化镜像文件的指令
$> mkinitramfs -o initrd-2.6.31-xen.img 2.6.31.13
# 或者是用这个指令(CentOS5)
$> mkinitrd -v -f --with=aacraid --with=sd_mod --with=scsi_mod initrd-2.6.31-xen.img 2.6.31.13

$> vi /boot/grub/menu.lst

title Xen 4.0, dom0 Linux kernel 2.6.31.13
kernel /boot/xen.gz dom0_mem=512M
module /boot/vmlinuz-2.6.31.13 root=/dev/sda7 ro nomodeset
module /boot/initrd-2.6.31-xen.img


注:一般内核编译中会有以下一些步骤:
$> make menuconfig
$> make
$> make bzImage
$> make modules
$> make modules_install
$> make install


References:
http://wiki.xensource.com/xenwiki/Xen4.0
http://wiki.xensource.com/xenwiki/XenParavirtOps

Friday, June 25, 2010

script标签中的defer属性说明

script中的defer属性默认情况下是false的,其主要作用是为了提升页面性能,实际在目前的HTML 4.0中这个属性是个鸡肋,在各浏览器中表现也不一样,最好忽略此属性。

微软MSDN中的文档说明摘录部分内容如下:
Remarks: Using the attribute at design time can improve the download performance of a page because the browser does not need to parse and execute the script and can continue downloading and parsing the page instead.
Standards Information: This property is defined in HTML 4.0 World Wide Web link and is defined in World Wide Web Consortium (W3C) Document Object Model (DOM) Level 1 World Wide Web link.

W3C中的说明摘录部分内容如下:
defer [CI]
When set, this boolean attribute provides a hint to the user agent that the script is not going to generate any document content (e.g., no "document.write" in javascript) and thus, the user agent can continue parsing and rendering.
指示脚本不会生成任何的文档内容(不要在其中使用document.write命令,不要在defer型脚本程序段中包括任何立即执行脚本要使用的全局变量或者函数),浏览器可以继续解析并绘制页面。但是defer的script在什么时候执行,执行顺序情况并无明确规定。

正在制定的HTML5有极大可能会完善script标签的定义,这里有简单的HTML5中defer属性的定义和用法

async和defer二个属性属性与src属性一起使用,async定义脚本是否异步执行。
如果 async 属性为 true,则脚本会相对于文档的其余部分异步执行,这样脚本会可以在页面继续解析的过程中来执行。
如果 async 属性为 false,而 defer 属性为 true,则脚本会在页面完成解析时得到执行。
如果 async 和 defer 属性均为 false,那么脚本会立即执行,页面会在脚本执行完毕继续解析。

The async and defer attributes are boolean attributes that indicate how the script should be executed.

There are three possible modes that can be selected using these attributes. If the async attribute is present, then the script will be executed asynchronously, as soon as it is available. If the async attribute is not present but the defer attribute is present, then the script is executed when the page has finished parsing. If neither attribute is present, then the script is fetched and executed immediately, before the user agent continues parsing the page. The exact processing details for these attributes are described below.

The defer attribute may be specified even if the async attribute is specified, to cause legacy Web browsers that only support defer (and not async) to fall back to the defer behavior instead of the synchronous blocking behavior that is the default.

If one or both of the defer and async attributes are specified, the src attribute must also be specified.

Sunday, June 20, 2010

relationship of constructor and prototype in javascript


var animal = function(status) {
this.status = status;
this.breathes = "yes";
this.action = function() {
console.log('flying...')
};
},
human = function() {
this.name = 'human';
},
cat = function() {
this.type = 'cat';
};

// javascript支持原型继承,这种方式比类继承更强大,类继承中一个对象可以继承结构和行为,而原型继承可以继承结构和行为之外,并可以继承一个对象的状态
// new一个animal的实例对象作为cat.prototype的原型,这个animal实例对象就成为cat的实例对象原型链上的一员
// __proto__这个魔法属性在这些浏览器不能工作: ie 6/7/8, safari < 5, opera < 10.50
//当在cat的某个实例上检索一个属性时,如果在其本身中没有找到,则会延着原型链向上检索,如下例子中的c.__proto__即为一个animal对象
//如果检索c.breathes,如果在c对象本身没有找到此属性,则会检索t.__proto__.breathes、t.__proto__.__proto__.breathes等原型链上的对象,直到找到为止,没找到返回undefined
cat.prototype = new animal("live");
//cat继承的原型对象是具有特定状态的animal对象
var c = new cat();
console.log(cat.prototype);
console.log(cat.prototype.constructor.tostring());
console.log(c.constructor.tostring());
console.log("cat breathes:" + c.breathes);
console.log("c.__proto__:", c.__proto__);
//ie不支持此属性
//你可以利用object.__proto__这个魔法属性修改当前对象的原型,下面将一只猫猫化为人形
var d = new cat();
d.__proto__ = new human();
console.log("d.__proto__:", d.__proto__);
//从上面结果可以看到cat的实例c.constructor不是指向cat这个构造函数,而是animal构造函数
//需要修改对象的constructor为其构造函数本身
//当一个函数对象被创建时,function构造器产生的函数对象会运行类似这样的一些代码:this.prototype = {constructor: this},参考javascript: the good parts 5.1节说明
//新函数对象被赋予一个prototype属性,其值是包含一个constuctor属性,并且其属性值为此新函数对象本身
//但是通过原型方式继承时,会给prototype重新赋予一个新对象,此prototype对象中的constructor是指向其自身的构造函数,而不是新函数的,所以需要重置其fn.prototype.constructor = this
//参考javascript权威指南第五版example 9-3. subclassing a javascript class
cat.prototype.constructor = cat;
console.log(cat.prototype.constructor.tostring());
console.log(c.constructor.tostring());
console.log(c.__proto__);
var tostring = object.prototype.tostring;
language = function() {
this.type = "programming";
return {
"locale": "en",
"class-free": function() {
return false
},
"tostring": function() {
return tostring.apply(this, arguments)
}
// 如果tostring方法被重写成非function对象,则后面console中无法输出对象j
}
},
javascript = function() {
this.value = "javascript";
this["class-free"] = function() {
return true
};
};
language.prototype = {
a: 1,
b: 2
};
javascript.prototype = new language();
var j = new javascript();
console.log(j);
console.log(j.__proto__);
//locale: en,此处因为language构造函数返回不是this,而是另一个object直接量,而object直接的构造方法为object(),因此language的原型被丢失了
console.log(language.prototype);
console.log(javascript.prototype);
console.log(j.constructor.tostring());
//function object() { [native code] }


构造函数与其返回值


构造函数会返回一个对象,如果没有直接return语句,构造函数会自动返回当前对象:"return this;",也可以返回一个对象直接量,而不返回this,这样会中断正常的原型链。

prototype.js中class对象定义是封装在一个匿名函数里的,从而使得其内部变量和方法与外界隔离,其中有二句代码为:

function subclass() {};
subclass.prototype = parent.prototype;

因为parent的构造可能返回语句不是返回this对象,而是返回了一个其他的对象,如{tostring:true},如果不用subclass.prototype=parent.prototype这样写,可能这样会丢失原型链上的方法和属性,通过subclass这个空构造将parent.prototype引用到自身的prototype上,从而保持住部分原型链。
这其实也已经不是原型继承了,因为它不是通过new parent()来获取原型对象,丢失了new parent所得对象中的属性和方法。
prototype中的class其实放弃了原型对象,只是简单的继承了parent.prototype对象,已经失去原型继承可以继承对象状态的功能,这样操作其实是很好的模似了类继承方式。

var class = (function() {
function subclass() {};
function create() {
var parent = null,
properties = $a(arguments);
if (object.isfunction(properties[0])) parent = properties.shift();

function klass() {
this.initialize.apply(this, arguments);
}

object.extend(klass, class.methods);
klass.superclass = parent;
klass.subclasses = [];

if (parent) {
// 因为parent的构造可能返回对象直接量,而不是返回this,如{tostring:true}
subclass.prototype = parent.prototype;
klass.prototype = new subclass;
parent.subclasses.push(klass);
}

for (var i = 0; i < properties.length; i++) klass.addmethods(properties[i]);
if (!klass.prototype.initialize) klass.prototype.initialize = prototype.emptyfunction;
klass.prototype.constructor = klass;
return klass;
}
function addmethods(source) {
var ancestor = this.superclass && this.superclass.prototype;
var properties = object.keys(source);
if (!object.keys({
tostring: true
}).length) {
if (source.tostring != object.prototype.tostring) properties.push("tostring");
if (source.valueof != object.prototype.valueof) properties.push("valueof");
}
for (var i = 0, length = properties.length; i < length; i++) {
var property = properties[i],
value = source[property];
if (ancestor && object.isfunction(value) && value.argumentnames().first() == "$super") {
var method = value;
value = (function(m) {
return function() {
return ancestor[m].apply(this, arguments);
};
})(property).wrap(method);
value.valueof = method.valueof.bind(method);
value.tostring = method.tostring.bind(method);
}
this.prototype[property] = value;
}
return this;
}
return {
create: create,
methods: {
addmethods: addmethods
}
};
})();

Tuesday, June 08, 2010

javascript: closures, lexical scope and scope chain

闭包的定义(javascript权威指南)如下:
JavaScript functions are a combination of code to be executed and the scope in which to execute them. This combination of code and scope is known as a closure in the computer science literature. All JavaScript functions are closures.
javascript的function定义了将要被执行的代码,并且指出在哪个作用域中执行这个方法,这种代码和作用域的组合体就是一个闭包,在代码中的变量是自由的未绑定的。闭包就像一个独立的生命体,有其自身要运行的代码,同时其自身携带了运行时所需要的环境。
javascript所有的function都是闭包。

闭包中包含了其代码运行的作用域,那这个作用域又是什么样子的呢,这就引入了词法作用域(lexical scope)的概念:
词法作用域是指方法运行的作用域是在方法定义时决定的,而不是方法运行时决定的。
所以在javascript中,function运行的作用域其实是一个static scope。但也有二个例外,就是with和eval,在这2者中的代码处于dynamic scope中,这给javascript带来额外的复杂度和计算量,因而也效率低下,避免使用。

当闭包在其词法作用域中运行过程中,如何检索其中的变量名?这就再引入了一个概念,作用域链(scope chain):
当一个方法function定义完成,其作用域链就是固定的了,并被保存成为方法内部状态的一部分,只是这个作用域链中调用对象的属性值不是固定的。作用域链是"活"的。
当一个方法在被调用时,会生成一个调用对象(call object or activation object),并将此call object加到其定义时确认下来的作用域链的顶端。
在这个call object上,方法的参数和方法内定义的局部变量名和值都会存在这个call object中,如果调用结束,这个call object会从作用域链的顶端移除,再没有被其他对象引用,内存也会被自动回收。
在此call object中使用的变量名会先从此方法局部变量和传入参数中检索,如果没有找到,就会向作用域链上的前一个对象查询,如此向上追溯,一直检索到global object(即window对象上),如果在整个作用域链上没有找到此变量名,则会返回undefined(没有指定对象直接查询变量名,没找到则抛出异常变量未定义)。
如此通过作用域链,javascrip就实现了call object中变量名检索。

在全局对象中一个方法调用完成之后,生成的call object会被回收,这看不出闭包(即当前被调用的方法)有什么功用。但是当一个外部方法的内部返回一个嵌套方法,并且返回的嵌套方法被全局对象引用时,或者是外部方法内将嵌套方法赋给全局对象的属性(jQuery构造方法就是在匿名方法内设置在window.jQuery上),外部方法调用生成的call object就会引用这个嵌套方法,而同时嵌套方法被全局对象引用,所以这个外部方法调用产生的call object及其属性就会继续生存在内存中,这时闭包(外部方法)的功用才被显示出来,下面以jQuery.fn.animation()方法调用过程为例进行说明:

1、当载入整个jquery.js文件时,会运行最外面的匿名方法(通过这个匿名方法形成一个命名空间,所有的变量名都是匿名方法内部定义的局部变量名):


(function( window, undefined ) {
// ......jQuery source code;
// Expose jQuery to the global object
window.jQuery = window.$ = jQuery;
})(window);
2、因为匿名方法内部有一个内部方法jQuery被全局对象window的属性jQuery和$引用,这里变量名很搞,一个是匿名方法内嵌套的构造方法jQuery,另一个window对象的属性名jQuery。因为这个匿名方法内部的jQuery构造方法被全局对象window.jQuery引用,所以外围的匿名方法在运行时产生的call object会继续生存在内存中。此时,这个call object可以利用Firebug或者Chrome的debug工具可以看到,在Firebug中的scopeChain中称之为"Object",在Chrome的console中称之为"Closure",该对象中记录了当前这个最外围的匿名方法被调用后生成的call object上变量的值,这些变量是未绑定的,是自由的,其值可以被修改并保存在作用域链上。运行此匿名方法时,会将其call object置于global object之上,形成作用域链。
这里注意一点,这匿名方法是一个闭包,但运行方法生成的call object对象只是作用域链顶端的一个对象,记录了方法中的变量名和值。闭包不但包括这个运行的作用域,还包括其运行所需的代码。
3、页面不关闭,这个匿名方法调用生成的call object就会一直驻在内存中,接下来当页面发生了一个jQuery.fn.animate()方法的调用,这个时候javascript又会为.animate()方法生成一个call object,这个对象拥有传进来的参数名和值,以及在.animate()方法内部定义的一个局部变量opt和它的值。
同时,javascript会将生成的这个call object置于其作用域链(scope chain)的最前端,即此时的作用域链为:global object->anonymous function call object->animate call object。
4、接下来会调用jQuery.fn.queue()->jQuery.fn.each()->jQuery.fn.dequeue(),在这些方法调用过程也都会接触到第2步中所提到的那个匿名方法调用后生成的闭包,这中间过程略过,当运行到最后传参给.queue(function)的function时,因为这个匿名方法是定义在jQuery.fn.animate()方法内部的,所以其作用域链(scope chain)也就已经确定了,即global object->anonymous function call object->animate call object,当此匿名方法调用生成一个call object,会将此call object再置于animate call object之上。
5、对于最后的匿名function运行完成之后,如果这个匿名function对象还被其他element的queue数组引用,则第3步中运行.animate()方法生成的闭包将继续生存在内存之中,直到所有的效果方法运行完成,此匿名function没有其他引用时,.animate()调用生成的call object就会被回收。

Reference: JavaScript函数调用时的作用域链和调用对象是如何形成的及与闭包的关系

Monday, May 24, 2010

mysqlbinlog Ver 3.3和3.1版本之间的输出差异

mysqlbinlog Ver 3.3中多了一个--base64-output参数,默认值为auto:


--base64-output[=name]
Determine when the output statements should be
base64-encoded BINLOG statements: 'never' disables it and
works only for binlogs without row-based events; 'auto'
is the default and prints base64 only when necessary
(i.e., for row-based events and format description
events); 'always' prints base64 whenever possible.
'always' is for debugging only and should not be used in
a production system. The default is 'auto'.
--base64-output is a short form for
--base64-output=always.

在原来的Ver3.1版本中没有此参数,对比mysql5.0生成的日志文件的输出结果:
Ver3.1版本中使用mysqlbinlog输出结果:

shell> mysqlbinlog mysql-bin.000182|less
/*!40019 SET @@session.max_insert_delayed_threads=0*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at 4
#100515 1:38:58 server id 25 end_log_pos 98 Start: binlog v 4, server v 5.0.37-log created 100515 1:38:58
# at 98

Ver3.3版本中使用mysqlbinlog输出结果:

shell> mysqlbinlog --base64-output=never mysql-bin.000182|less
/*!40019 SET @@session.max_insert_delayed_threads=0*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at 4
#100515 1:38:58 server id 25 end_log_pos 98 Start: binlog v 4, server v 5.0.37-log created 100515 1:38:58
# at 98

shell> mysqlbinlog --base64-output=auto mysql-bin.000182|less
/*!40019 SET @@session.max_insert_delayed_threads=0*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at 4
#100515 1:38:58 server id 25 end_log_pos 98 Start: binlog v 4, server v 5.0.37-log created 100515 1:38:58
BINLOG '
sortSw8ZAAAAXgAAAGIAAAAAAAQANS4wLjM3LWxvZwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAEzgNAAgAEgAEBAQEEgAASwAEGg==
'/*!*/;
# at 98

从上面结果比较而言,相当于3.1版本的mysqlbinlog的base64-output参数默认为never。
在mysql 5.0中带的mysqlbinlog一般为3.1版本的,而mysql5.1版本中带的mysqlbinlog则为3.3版本,如果需要将mysql5.0生成的日志文件导入mysql5.1时,需要将mysqlbinlog的base64-output值设置为never。如果不设置此参数,导入日志时,会在
BINLOG 'sortSw8ZAAAAXgAAAGIAAAAAAAQANS4wLjM3LWxvZwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAEzgNAAgAEgAEBAQEEgAASwAEGg=='

处报SQL语法错误。

MySQL5 slave同步报错(server_errno=1236)

100519 19:35:33 [Note] Slave I/O thread: connected to master 'repl@master:3306', replication started in log 'mysql-bin.000176' at position 615525147

100519 19:35:33 [ERROR] Error reading packet from server: Could not find first log file name in binary log index file ( server_errno=1236)

从日志信息上可知,slave已经连接到master,并且准备从指定的binlog文件指定位置开始同步,但后面错误提示日志文件找不到,而master服务器上日志文件是存在的。
这种情况有二种处理方法:
一、重启主数据库(master)之后,然后slave上stop slave;start slave,再检查同步的状态。
二、不重启主服务器,则用mysqlbinlog根据start-position或者start-datetime,将日志分析出来后,将分析结果在slave上用mysql命令导入,导入完成后,再用CHANGE MASTER TO语句,从下一个日志文件MASTER_LOG_POSITION=98开始同步。

另外在使用mysqlbinlog工具进行日志导入时,需要注意以下问题,下面内容转自MySQL官方手册:
如果MySQL服务器上有多个要执行的二进制日志,安全的方法是在一个连接中处理它们。下面是一个说明什么是不安全的例子:

shell> mysqlbinlog hostname-bin.000001 | mysql # DANGER!!
shell> mysqlbinlog hostname-bin.000002 | mysql # DANGER!!

使用与服务器的不同连接来处理二进制日志时,如果第1个日志文件包含一个CREATE TEMPORARY TABLE语句,第2个日志包含一个使用该临时表的语句,则会造成问题。当第1个mysql进程结束时,服务器撤销临时表。当第2个mysql进程想使用该表时,服务器报告 “不知道该表”。

要想避免此类问题,使用一个连接来执行想要处理的所有二进制日志中的内容。下面提供了一种方法:

shell> mysqlbinlog hostname-bin.000001 hostname-bin.000002 | mysql -u root -ppassword

Reference: http://yuweijun.blogspot.com/2009/12/mysql50.html

javascript中keyEvent按键事件说明(录自javascript权威指南第5版)

有3种按键类型,分别是keydown、keypress和keyup,它们分别对应onkeydown、onkeypress和onkeyup这几个事件处理器。
一个按键操作会产生这3个事件,依次是keydown、keypress,然后在按键释放的时候keyup。
这3个事件类型中,keypress事件是最为用户友好的:和它们相关的事件对象包含了所产生的实际字符的编码。keydown和keyup事件是较底层的,它们的按键事件包含一个和键盘所生成的硬件编码相关的“虚拟按键码”。对于ASCII字符集中的数字和字符,这些虚拟按键码和ASCII码相同。如果按下SHIFT键并按下数字2,keydown事件将通知发生了“SHITF-2”的按键事件。keypress事件会解释这一事件,说明这次按键产生了一个可打印的字符“@”。
对于不能打印的功能按键,如Backspace、Enter、Escape和箭头方向键、Page Up、Page Down以及F1到F12,它们会产生keydown和keyup事件。

在不同的浏览器中,按键事件的一些细节区别如下:


1、对于不能打印的功能按键,在Firefox中,也会产生keypress事件,在IE和Chrome中,则不会触发keypress事件,只有当按键有一个ASCII码的时候,即此字符为可打印字符或者一个控制字符的时候,keypress事件才会发生。对于这些不能打印的功能按键,可通过和keydown事件相关的keyCode来获取。
2、作为一条通用的规则,keydown事件对于功能按键来说是最有用的,而keypress事件对于可打印的按键来说是最有用的。
3、在IE中,Alt按键组合被认为是无法打印的,所以并不会触发keypress事件。
4、在Firefox中,按键事件定义有二个属性,keyCode存储了一个按键的较低层次的虚拟按键码,并且和keydown事件一起发送。charCode存储了按下一个键时所产生的可打印的字符的编码,并且和keypress事件一起发送。在Firefox中,功能按键会产生一个keypress事件,在这种情况下,charCode是0,而keyCode包含了虚拟按键码。在Firefox中,发生keydown事件时,charCode都为0,所以在keydown时获取charCode是无意义的。
5、在IE中,只有一个keyCode属性,并且它的解释也取决于事件的类型。对于keydown事件来说,keyCode是一个虚拟按键码,对于keypress事件来说,keyCode是一个字符码。
6、在Chrome中,功能键与IE中表现一样,不会触发keypress事件,对于keydown事件,也会在事件的keyCode中存储虚拟按键码,而charCode为0,与IE和Firefox表现一样,然而在发生可打印字符的keypress事件时,除了与Firefox一样,会在事件的charCode中存储实际按键编码之外,也会在keyCode中存储实际按键码,这二个值相同。
7、charCode字符码可以使用静态方数String.fromCharCode()转为字符。

rails session options

require 'action_controller/cgi_ext'
require 'action_controller/session/cookie_store'

module ActionController #:nodoc:
class Base
# Process a request extracted from a CGI object and return a response. Pass false as session_options to disable
# sessions (large performance increase if sessions are not needed). The session_options are the same as for CGI::Session:
#
# * :database_manager - standard options are CGI::Session::FileStore, CGI::Session::MemoryStore, and CGI::Session::PStore
# (default). Additionally, there is CGI::Session::DRbStore and CGI::Session::ActiveRecordStore. Read more about these in
# lib/action_controller/session.
# * :session_key - the parameter name used for the session id. Defaults to '_session_id'.
# * :session_id - the session id to use. If not provided, then it is retrieved from the +session_key+ cookie, or
# automatically generated for a new session.
# * :new_session - if true, force creation of a new session. If not set, a new session is only created if none currently
# exists. If false, a new session is never created, and if none currently exists and the +session_id+ option is not set,
# an ArgumentError is raised.
# * :session_expires - the time the current session expires, as a Time object. If not set, the session will continue
# indefinitely.
# * :session_domain - the hostname domain for which this session is valid. If not set, defaults to the hostname of the
# server.
# * :session_secure - if +true+, this session will only work over HTTPS.
# * :session_path - the path for which this session applies. Defaults to the directory of the CGI script.
# * :cookie_only - if +true+ (the default), session IDs will only be accepted from cookies and not from
# the query string or POST parameters. This protects against session fixation attacks.
def self.process_cgi(cgi = CGI.new, session_options = {})
new.process_cgi(cgi, session_options)
end

def process_cgi(cgi, session_options = {}) #:nodoc:
process(CgiRequest.new(cgi, session_options), CgiResponse.new(cgi)).out
end
end

end

Regexp Magic in vim

Some characters in the pattern are taken literally. They match with the same
character in the text. When preceded with a backslash however, these
characters get a special meaning.

Other characters have a special meaning without a backslash. They need to be
preceded with a backslash to match literally.

If a character is taken literally or not depends on the 'magic' option and the
items mentioned next.

*/\m* */\M*
Use of "\m" makes the pattern after it be interpreted as if 'magic' is set,
ignoring the actual value of the 'magic' option.
Use of "\M" makes the pattern after it be interpreted as if 'nomagic' is used.

*/\v* */\V*
Use of "\v" means that in the pattern after it all ASCII characters except
'0'-'9', 'a'-'z', 'A'-'Z' and '_' have a special meaning. "very magic"

Use of "\V" means that in the pattern after it only the backslash has a
special meaning. "very nomagic"

Examples:


after: \v \m \M \V matches ~
$ $ $ \$ matches end-of-line
. . \. \. matches any character
* * \* \* any number of the previous atom
() \(\) \(\) \(\) grouping into an atom
| \| \| \| separating alternatives
\a \a \a \a alphabetic character
\\ \\ \\ \\ literal backslash
\. \. . . literal dot
\{ { { { literal '{'
a a a a literal 'a'

{only Vim supports \m, \M, \v and \V}


It is recommended to always keep the 'magic' option at the default setting,
which is 'magic'. This avoids portability problems. To make a pattern immune
to the 'magic' option being set or not, put "\m" or "\M" at the start of the
pattern.

Without \v:
:%s/^\%(foo\)\{1,3}\(.\+\)bar$/\1/

With \v:
:%s/\v^%(foo){1,3}(.+)bar$/\1/

See also :h /\v

在CentOS 5中开启apache2的mod_ssl模块

在配置完/etc/httpd/conf.d/ssl.conf文件之后,进行配置文件语法正确性测试时,报语法错误如下:
$> httpd -t
Syntax error on line 5 of /etc/httpd/conf.d/ssl.conf:
Invalid command 'SSLPassPhraseDialog', perhaps misspelled or defined by a module not included in the server configuration

移除ssl.conf文件之后,语法检查正确,列出httpd的模块检查:
$> httpd -M
Loaded Modules:
core_module (static)
mpm_prefork_module (static)
http_module (static)
so_module (static)
auth_basic_module (shared)
auth_digest_module (shared)
authn_file_module (shared)
authn_alias_module (shared)
authn_anon_module (shared)
authn_dbm_module (shared)
authn_default_module (shared)
authz_host_module (shared)
authz_user_module (shared)
authz_owner_module (shared)
authz_groupfile_module (shared)
authz_dbm_module (shared)
authz_default_module (shared)
ldap_module (shared)
authnz_ldap_module (shared)
include_module (shared)
log_config_module (shared)
logio_module (shared)
env_module (shared)
ext_filter_module (shared)
mime_magic_module (shared)
expires_module (shared)
deflate_module (shared)
headers_module (shared)
usertrack_module (shared)
setenvif_module (shared)
mime_module (shared)
dav_module (shared)
status_module (shared)
autoindex_module (shared)
info_module (shared)
dav_fs_module (shared)
vhost_alias_module (shared)
negotiation_module (shared)
dir_module (shared)
actions_module (shared)
speling_module (shared)
userdir_module (shared)
alias_module (shared)
rewrite_module (shared)
proxy_module (shared)
proxy_balancer_module (shared)
proxy_ftp_module (shared)
proxy_http_module (shared)
proxy_connect_module (shared)
cache_module (shared)
suexec_module (shared)
disk_cache_module (shared)
file_cache_module (shared)
mem_cache_module (shared)
cgi_module (shared)
version_module (shared)
proxy_ajp_module (shared)
Syntax OK

发现apache没有加载ssl_module,并且/etc/httpd/modules目录中无mod_ssl.so文件,需要在线安装:
$> yum search mod_ssl
mod_ssl.x86_64 : SSL/TLS module for the Apache HTTP server

$> yum install mod_ssl
Installed: mod_ssl.x86_64 1:2.2.3-43.el5.centos
Dependency Installed: distcache.x86_64 0:1.4.5-14.1
Updated: httpd.x86_64 0:2.2.3-43.el5.centos
Complete!

安装mod_ssl会增加一个用户类型,所以会修改/etc/passwd文件。
安装完成之后,修改ssl.conf文件,在文件顶部添加以下一行代码,加载ssl_module:

LoadModule ssl_module modules/mod_ssl.so
Listen 443
AddType application/x-x509-ca-cert .crt
AddType application/x-pkcs7-crl .crl
SSLPassPhraseDialog builtin
SSLSessionCache shmcb:/var/cache/mod_ssl/scache(512000)
SSLSessionCacheTimeout 300
SSLMutex default

<VirtualHost _default_:443>
DocumentRoot "/path/to/wwwroot"
ServerName www.test.com:443
ServerAdmin test.yu@gmail.com
SSLEngine on
SSLCipherSuite ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM:+LOW
SSLCertificateFile /usr/local/apache2/conf/ssl.crt/server.crt
SSLCertificateKeyFile /usr/local/apache2/conf/ssl.key/server.key
SSLCACertificateFile /usr/local/apache2/conf/ssl.crt/cacertificate.crt
<FilesMatch "\.(cgi|shtml|phtml|php)$">
SSLOptions +StdEnvVars
</FilesMatch>
<Directory "/usr/local/apache2/cgi-bin">
SSLOptions +StdEnvVars
</Directory>
SetEnvIf User-Agent ".*MSIE.*" \
nokeepalive ssl-unclean-shutdown \
downgrade-1.0 force-response-1.0
</VirtualHost>
再测试httpd配置文件:
$> httpd -t
Syntax OK

MySQL: Many tables or many databases?

Question:
For a project we having a bunch of data that always have the same structure and is not linked together. There are two approaches to save the data:
* Creating a new database for every pool (about 15-25 tables)
* Creating all the tables in one database and differ the pools by table names.
Which one is easier and faster to handle for MySQL?

Answer:
There should be no significant performance difference between multiple tables in a single database versus multiple tables in separate databases.

In MySQL, databases (standard SQL uses the term "schema" for this) serve chiefly as a namespace for tables. A database has only a few attributes, e.g. the default character set and collation. And that usage of GRANT makes it convenient to control access privileges per database, but that has nothing to do with performance.

You can access tables in any database from a single connection (provided they are managed by the same instance of MySQL Server). You just have to qualify the table name:

SELECT * FROM database17.accounts_table;

This is purely a syntactical difference. It should have no effect on performance.

Regarding storage, you can't organize tables into a file-per-database as @Chris speculates. With the MyISAM storage engine, you always have a file per table. With the InnoDB storage engine, you either have a single set of storage files that amalgamate all tables, or else you have a file per table (this is configured for the whole MySQL server, not per database). In either case, there's no performance advantage or disadvantage to creating the tables in a single database versus many databases.

There aren't many MySQL configuration parameters that work per database. Most parameters that affect server performance are server-wide in scope.

Regarding backups, you can specify a subset of tables as arguments to the mysqldump command. It may be more convenient to back up logical sets of tables per database, without having to name all the tables on the command-line. But it should make no difference to performance, only convenience for you as you enter the backup command.

-- Bill Karwin (the author of SQL Antipatterns from Pragmatic Bookshelf)

linux命令行显示指定行号的内容

以第四行为例,要查询的文件名为list.txt:
方法1:

grep -n '^' list.txt |grep '^4:'|grep -o '[^4:].*'

方法2:
sed -n '4p' list.txt

sed -n '4,4p' list.txt
方法3:
awk '{if ( NR==4 ) print $0}' list.txt

方法4:
tac list.txt |tail -4|tac|tail -1
tac list.txt |tail -n 4|tac|tail -n 1

References: http://zhidao.baidu.com/question/91856742

Tuesday, May 11, 2010

利用document.write()和noscript标签做内文替换的小技巧

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>noscript tips</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
</head>
<body>
<script>document.write("新页面内容" + "<nosc"+"ript>");</script>
原来的页面内容: 利用document.write()打印新的页面内容,并且打出一个noscript起始标签,在新页面内容想要替换的页面内容结尾处,添加一个noscript结束标签,从而利用document.write()和noscript标签对,用新页面内容替换原来的页面内容,这个小技巧是在看google做页面内容不同时的效果跟踪代码时发现的。
</noscript>
</body>
</html>

Tuesday, April 20, 2010

IE6中select组件与css中z-index的冲突

Positioned Elements and OS Controls, Applets and Plug-ins这篇文章中,描述了当position为absolute一个DIV被拖到select组件上方的时候,在IE6中,select组件会透过DIV层显示出来,这并不是想要的效果,这个问题并非是一个bug,因为在IE6中,select元素是一个window控制组件,并不支持z-index属性,在IE7之后则开始支持z-index属性,更详细的说明可查看MSDN关于select的文档:
From Microsoft Internet Explorer 5 to Internet Explorer 6, This element is a windowed control and does not support the z-index attribute or zIndex property.
As of Internet Explorer 7, this element is windowless and supports the z-index attribute and the zIndex property. The SELECT element does not require a strict doctype to enable windowless functionality.
This element is an inline element.
This element requires a closing tag.

Friday, March 26, 2010

notice "var" keywords of javascript in IE8

当前测试只是针对IE8,没有测试之前IE版本,以下代码在firefox/chrome中会按预期一样正常运行,代码测试注意打开firebug或者是console查看输出:


<script type="text/javascript">
// window.a 设置一个属性a到window上
window.a = 1;
// 正常打印出a:1
window.console.log(a);
</script>



<script type="text/javascript">
// 必须在此分二块script来写,如果写在一块script标签内,都会正常执行
// 在同一个script代码块内,javascript会解析当前代码块中所有的变量(之前没有出现过的变量名),并且都为 undefined
// 在这里还没有执行到var a=2,a为 undefined
// 针对IE8: window.a 也被置为 undefined
// 其他浏览器: window.a前面代码中已经定义,所以这里还是原来的值:1,而非像IE8中一样为 undefined
window.console.log(a);
var a = 2; // 注意前面有个变量声明: var
window.console.log(a);
</script>



<script type="text/javascript">
window.console.log(a);
// 这里因为没有用var新定义变量,所以a即为window.a,只是对变量重新赋值,代码执行没有问题
a = 3; // 注意前面没有变量声明: var
window.console.log(a);
</script>

以上代码在firefox/chrome中执行结果为: 1 1 2 2 3
以上代码在IE8中执行结果为: 1 undefined 2 2 3

隐藏ubuntu命令行中运行gvim的警告信息

$> gvim
** (gvim:6676): CRITICAL **: gtk_form_set_static_gravity: assertion `static_gravity_supported' failed
** (gvim:6676): CRITICAL **: gtk_form_set_static_gravity: assertion `static_gravity_supported' failed
** (gvim:6676): CRITICAL **: gtk_form_set_static_gravity: assertion `static_gravity_supported' failed
** (gvim:6676): CRITICAL **: gtk_form_set_static_gravity: assertion `static_gravity_supported' failed
** (gvim:6676): CRITICAL **: gtk_form_set_static_gravity: assertion `static_gravity_supported' failed

每次运行看到这些警告很不爽,网上找了一下,发现是一个待修正的问题,暂时只能将错误重定向到个人目录的.xsession-errors中,重写一个gvim的命令,放在个人主目录~/bin下:

#!/bin/bash
## Execute gvim, sending X/gtk errors to the standard place.
exec /usr/bin/gvim $* > /dev/null 2>>$HOME/.xsession-errors

Tuesday, March 09, 2010

一个tomcat启动二个不同端口的web服务

首先在server.xml模仿Service Catalina一样再配一个Service,修改Service/Engine的name,Connector的port

<Service name="Catalina2">
<Connector port="8081" />
<Engine name="Catalina2" defaultHost="localhost">
<Realm className="org.apache.catalina.realm.UserDatabaseRealm"
resourceName="UserDatabase"/>
<Host name="localhost" appBase="webapps"
unpackWARs="true" autoDeploy="true"
xmlValidation="false" xmlNamespaceAware="false">
</Host>
</Engine>
</Service>
其次复制conf下的Catalina整个目录为新的Service名字,注意里面localhost文件夹里的配置因项目不同,进行调整:
$> cp -r conf/Catalina conf/Catalina2