Linux – Page 4 – Neohope's Blog

Hadoop Linux Native 编译说明

Posted on 2015/10/09 by neohope — No Comments ↓

首先说明一下，如果要使用Linux Native的话，Hadoop是已经自带了哦

然后，如果要编译的话，建议直接从Hadoop源码按官方的说明进行编译，不要像我这样自己搞。。。

如果你喜欢折腾，请继续看：

1、按源码架构拷贝下面的文件及文件夹

hadoop-2.5.2-src\hadoop-common-project\hadoop-common\src\main\native
hadoop-2.5.2-src\hadoop-common-project\hadoop-common\src\CMakeLists.txt
hadoop-2.5.2-src\hadoop-common-project\hadoop-common\src\config.h.cmake
hadoop-2.5.2-src\hadoop-common-project\hadoop-common\src\JNIFlags.cmake
hadoop-2.5.2-src\hadoop-hdfs-project\hadoop-hdfs\src\main\native
hadoop-2.5.2-src\hadoop-hdfs-project\hadoop-hdfs\src\CMakeLists.txt（可能需要调整一下依赖文件JNIFlags.cmake的相对路径）
hadoop-2.5.2-src\hadoop-hdfs-project\hadoop-hdfs\src\config.h.cmake

2、编译libhadoop
2.1、检查并安装以来关系

#需要gcc、make、jdk，这些一般大家都有了
#需要zlib
apt-get install zlib1g-dev
#需要cmake
apt-get install cmake

2.2、用cmake生成Makefile

cmake ./src/ -DGENERATED_JAVAH=~/Build/hadoop-2.5.2-src/build/hadoop-common-project/hadoop-common/native/javah -DJVM_ARCH_DATA_MODEL=64 -DREQUIRE_BZIP2=false -DREQUIRE_SNAPPY=false

2.3、用javah生成头文件
需要三个jar包，hadoop-common,hadoop-annotations,guava

javah org.apache.hadoop.io.compress.lz4.Lz4Compressor
javah org.apache.hadoop.io.compress.lz4.Lz4Decompressor
javah org.apache.hadoop.io.compress.zlib.ZlibCompressor
javah org.apache.hadoop.io.compress.zlib.ZlibDecompressor
javah org.apache.hadoop.io.nativeio.NativeIO 
javah org.apache.hadoop.io.nativeio.SharedFileDescriptorFactory
javah org.apache.hadoop.net.unix.DomainSocket
javah org.apache.hadoop.net.unix.DomainSocketWatcher
javah org.apache.hadoop.security.JniBasedUnixGroupsMapping
javah org.apache.hadoop.security.JniBasedUnixNetgroupsMapping
javah org.apache.hadoop.util.NativeCrc32

将生成的文件，拷贝到对应的c文件夹中

2.3、生成

make

3、编译libhdfs
3.1、用cmake生成Makefile

cmake ./src/ -DGENERATED_JAVAH=~/Build/hadoop-2.5.2-src/build/hadoop-common-project/hadoop-common/native/javah -DJVM_ARCH_DATA_MODEL=64 -DREQUIRE_LIBWEBHDFS=false -DREQUIRE_FUSE=false

3.2、生成

make

4、将生成的文件拷贝到HADOOP_HOME/lib/mynative

5、修改/etc/profile，增加下面一行

export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/mynative"

6、刷新配置

source /etc/profile

搞定！

Debian编译Ruby

Posted on 2014/10/16 by neohope — No Comments ↓

1、首先要安装两个依赖项

#apt-get install zlib1g zlib1g-dev
#apt-get install libssl libssl-dev

2、然后下载Ruby源码
https://www.ruby-lang.org/en/

./configure --prefix=usr/ruby/ruby_2.2.3
make
make install

3、配置环境变量
编辑/etc/profile

export RUBY_HOME=/usr/ruby/ruby_2.2.3
export RUBYLIB=$RUBY_HOME/lib/ruby/2.2.0:$RUBY_HOME/lib/ruby/2.2.0/x86_64-linux:$RUBY_HOME/share/ri/2.2.0:$RUBYLIB
export PATH=$RUBY_HOME/bin:$PATH

更新环境变量

source /etc/profile

4、切换为国内源

gem sources --add https://ruby.taobao.org/
gem sources --remove https://rubygems.org/
gem sources -l

5、安装rails

$ gem install rails

生成xorg.conf文件

Posted on 2012/11/04 by neohope — No Comments ↓

1、生成xorg.conf文件

#如果有必要，停止gdm3
service gdm3 stop
#生成空白文件
Xorg -configure
#移动文件
mv ~/xorg.conf.new /etc/X11/xorg.conf
#修改文件，增加需要的分辨率
#如果有必要，开启gdm3
service gdm3 start

2、xorg.conf.new文件

Section "ServerLayout"
	Identifier     "X.org Configured"
	Screen      0  "Screen0" 0 0
	InputDevice    "Mouse0" "CorePointer"
	InputDevice    "Keyboard0" "CoreKeyboard"
EndSection

Section "Files"
	ModulePath   "/usr/lib/xorg/modules"
	FontPath     "/usr/share/fonts/X11/misc"
	FontPath     "/usr/share/fonts/X11/cyrillic"
	FontPath     "/usr/share/fonts/X11/100dpi/:unscaled"
	FontPath     "/usr/share/fonts/X11/75dpi/:unscaled"
	FontPath     "/usr/share/fonts/X11/Type1"
	FontPath     "/usr/share/fonts/X11/100dpi"
	FontPath     "/usr/share/fonts/X11/75dpi"
	FontPath     "/var/lib/defoma/x-ttcidfont-conf.d/dirs/TrueType"
	FontPath     "built-ins"
EndSection

Section "Module"
	Load  "record"
	Load  "glx"
	Load  "extmod"
	Load  "dbe"
	Load  "dri"
	Load  "dri2"
EndSection

Section "InputDevice"
	Identifier  "Keyboard0"
	Driver      "kbd"
EndSection

Section "InputDevice"
	Identifier  "Mouse0"
	Driver      "mouse"
	Option	    "Protocol" "auto"
	Option	    "Device" "/dev/input/mice"
	Option	    "ZAxisMapping" "4 5 6 7"
EndSection

Section "Monitor"
	Identifier   "Monitor0"
	VendorName   "Monitor Vendor"
	ModelName    "Monitor Model"
EndSection

Section "Device"
        ### Available Driver options are:-
        ### Values: <i>: integer, <f>: float, <bool>: "True"/"False",
        ### <string>: "String", <freq>: "<f> Hz/kHz/MHz"
        ### [arg]: arg optional
	Identifier  "Card0"
	Driver      "vboxvideo"
	VendorName  "InnoTek Systemberatung GmbH"
	BoardName   "VirtualBox Graphics Adapter"
	BusID       "PCI:0:2:0"
EndSection

Section "Screen"
	Identifier "Screen0"
	Device     "Card0"
	Monitor    "Monitor0"
	SubSection "Display"
		Viewport   0 0
		Depth     1
	EndSubSection
	SubSection "Display"
		Viewport   0 0
		Depth     4
	EndSubSection
	SubSection "Display"
		Viewport   0 0
		Depth     8
	EndSubSection
	SubSection "Display"
		Viewport   0 0
		Depth     15
	EndSubSection
	SubSection "Display"
		Viewport   0 0
		Depth     16
	EndSubSection
	SubSection "Display"
		Viewport   0 0
		Depth     24
	EndSubSection
EndSection

3、新版xorg.conf文件

Section "ServerLayout"
	Identifier     "X.org Configured"
	Screen      0  "Screen0" 0 0
	InputDevice    "Mouse0" "CorePointer"
	InputDevice    "Keyboard0" "CoreKeyboard"
EndSection

Section "Files"
	ModulePath   "/usr/lib/xorg/modules"
	FontPath     "/usr/share/fonts/X11/misc"
	FontPath     "/usr/share/fonts/X11/cyrillic"
	FontPath     "/usr/share/fonts/X11/100dpi/:unscaled"
	FontPath     "/usr/share/fonts/X11/75dpi/:unscaled"
	FontPath     "/usr/share/fonts/X11/Type1"
	FontPath     "/usr/share/fonts/X11/100dpi"
	FontPath     "/usr/share/fonts/X11/75dpi"
	FontPath     "/var/lib/defoma/x-ttcidfont-conf.d/dirs/TrueType"
	FontPath     "built-ins"
EndSection

Section "Module"
	Load  "record"
	Load  "glx"
	Load  "extmod"
	Load  "dbe"
	Load  "dri"
	Load  "dri2"
EndSection

Section "InputDevice"
	Identifier  "Keyboard0"
	Driver      "kbd"
EndSection

Section "InputDevice"
	Identifier  "Mouse0"
	Driver      "mouse"
	Option	    "Protocol" "auto"
	Option	    "Device" "/dev/input/mice"
	Option	    "ZAxisMapping" "4 5 6 7"
EndSection

Section "Monitor"
	Identifier   "Monitor0"
	VendorName   "Monitor Vendor"
	ModelName    "Monitor Model"
EndSection

Section "Device"
        ### Available Driver options are:-
        ### Values: <i>: integer, <f>: float, <bool>: "True"/"False",
        ### <string>: "String", <freq>: "<f> Hz/kHz/MHz"
        ### [arg]: arg optional
	Identifier  "Card0"
	Driver      "vboxvideo"
	VendorName  "InnoTek Systemberatung GmbH"
	BoardName   "VirtualBox Graphics Adapter"
	BusID       "PCI:0:2:0"
EndSection

Section "Screen"
	Identifier "Screen0"
	Device     "Card0"
	Monitor    "Monitor0"
	SubSection "Display"
		Viewport   0 0
		Depth     1
		Modes	"1280x800" "1024x768" "800x600"
	EndSubSection
	SubSection "Display"
		Viewport   0 0
		Depth     4
		Modes	"1280x800" "1024x768" "800x600"
	EndSubSection
	SubSection "Display"
		Viewport   0 0
		Depth     8
		Modes	"1280x800" "1024x768" "800x600"
	EndSubSection
	SubSection "Display"
		Viewport   0 0
		Depth     15
		Modes	"1280x800" "1024x768" "800x600"
	EndSubSection
	SubSection "Display"
		Viewport   0 0
		Depth     16
		Modes	"1280x800" "1024x768" "800x600"
	EndSubSection
	SubSection "Display"
		Viewport   0 0
		Depth     24
		Modes	"1280x800" "1024x768" "800x600"
	EndSubSection
EndSection

Debian6 Squeeze修改Terminal分辨率

Posted on 2012/11/03 by neohope — No Comments ↓

试了不少方法，要么参数不支持，要不设置了没有用。

好吧，简单暴力一些。

编辑/boot/grub/grub.cfg，在启动参数前，增加两行：

### BEGIN /etc/grub.d/10_linux ###
menuentry 'Debian GNU/Linux, with Linux 2.6.32-5-amd64' --class debian --class gnu-linux --class gnu --class os {
	#第一行>>>>>>set gfxpayload=1024x768x24
	#第二行>>>>>>load_video
	insmod part_msdos
	insmod ext2
	set root='(hd0,msdos1)'
	search --no-floppy --fs-uuid --set a600ccdc-3768-464b-9b19-29ec051f93e5
	echo	'Loading Linux 2.6.32-5-amd64 ...'
	linux	/boot/vmlinuz-2.6.32-5-amd64 root=UUID=a600ccdc-3768-464b-9b19-29ec051f93e5 ro  quiet text
	echo	'Loading initial ramdisk ...'
	initrd	/boot/initrd.img-2.6.32-5-amd64
}
menuentry 'Debian GNU/Linux, with Linux 2.6.32-5-amd64 (recovery mode)' --class debian --class gnu-linux --class gnu --class os {
	set gfxpayload=1024x768
	insmod part_msdos
	insmod ext2
	set root='(hd0,msdos1)'
	search --no-floppy --fs-uuid --set a600ccdc-3768-464b-9b19-29ec051f93e5
	echo	'Loading Linux 2.6.32-5-amd64 ...'
	linux	/boot/vmlinuz-2.6.32-5-amd64 root=UUID=a600ccdc-3768-464b-9b19-29ec051f93e5 ro single 
	echo	'Loading initial ramdisk ...'
	initrd	/boot/initrd.img-2.6.32-5-amd64
}
### END /etc/grub.d/10_linux ###

修改后为：

### BEGIN /etc/grub.d/10_linux ###
menuentry 'Debian GNU/Linux, with Linux 2.6.32-5-amd64' --class debian --class gnu-linux --class gnu --class os {
	set gfxpayload=1024x768x24
	load_video
	insmod part_msdos
	insmod ext2
	set root='(hd0,msdos1)'
	search --no-floppy --fs-uuid --set a600ccdc-3768-464b-9b19-29ec051f93e5
	echo	'Loading Linux 2.6.32-5-amd64 ...'
	linux	/boot/vmlinuz-2.6.32-5-amd64 root=UUID=a600ccdc-3768-464b-9b19-29ec051f93e5 ro  quiet text
	echo	'Loading initial ramdisk ...'
	initrd	/boot/initrd.img-2.6.32-5-amd64
}
menuentry 'Debian GNU/Linux, with Linux 2.6.32-5-amd64 (recovery mode)' --class debian --class gnu-linux --class gnu --class os {
	set gfxpayload=1024x768
	insmod part_msdos
	insmod ext2
	set root='(hd0,msdos1)'
	search --no-floppy --fs-uuid --set a600ccdc-3768-464b-9b19-29ec051f93e5
	echo	'Loading Linux 2.6.32-5-amd64 ...'
	linux	/boot/vmlinuz-2.6.32-5-amd64 root=UUID=a600ccdc-3768-464b-9b19-29ec051f93e5 ro single 
	echo	'Loading initial ramdisk ...'
	initrd	/boot/initrd.img-2.6.32-5-amd64
}
### END /etc/grub.d/10_linux ###

Debian6报错mptctl模块丢失

Posted on 2012/11/03 by neohope — No Comments ↓

1、查看设备是否存在

#执行发现设备不存在
ls /dev | grep mpt
mpt-status
#新建设备
mknod /dev/mptctl c 10 220

2、手动加载模块

modeprobe mptctl
lsmod | grep mpt

3、自动加载模块
编辑/etc/moduels，增加一行

mptctl

4、重启搞定

用户增加到sudoers文件

Posted on 2012/10/10 by neohope — No Comments ↓

使用超级用户，运行visudo命令，编辑sudoers列表，新增一行即可：

root	ALL=(ALL) ALL
user01	ALL=(ALL) ALL

Linux及BSD包管理系统

Posted on 2012/04/28 by neohope — No Comments ↓

大家都知道，
Linux发行版=Linux内核+包管理系统

当下，Linux的主要包管理系统分为两大阵营，RPM和DEB，
（其主要原因是，子发行版多沿用了父发行版的包管理系统）
但每种有特色的发行版，又都会开发自己独特的包管理系统。
主要Linux发行版的管理统系统计如下：

RPM:
RHEL;Fedora;CentOS;
openSUSE;SUSE Linux Enterprise;
PCLinuxOS;
Mageia;
Fuduntu;
Mandriva Linux;
Scientific Linux;
Oracle Linux;

DEB:
Debian;
Ubuntu;Kubuntu;Xubuntu;Ubuntu Studio;
Lubuntu;
Linux Mint;
Ultimate Edition;
Bodhi Linux;
Pinguy OS;
Zorin OS;
MEPIS Linux;
KNOPPIX;
BackTrack;

Pacman:
Arch Linux;
Chakra GNU/Linux;

Portage:
Gentoo

PET:
Puppy

TCE:
Tiny Core Linux

Entropy:
Sabayon Linux

TXZ:
Slackware Linux

TBZ:
FreeBSD;
DragonFly BSD;

TGZ:
OpenBSD

PBI/TGZ:
PC-BSD

IPS, pkgadm:
Oracle Solaris

pkgin:
MINIX

APT设置代理

Posted on 2012/01/29 by neohope — No Comments ↓

修改配置文件/etc/apt/apt.conf，并增加以下内容：

Acquire::http::proxy "http://uid:pwd@ip:port/";
Acquire::https::proxy "https://uid:pwd@ip:port/";