
在编程领域,编译语言凭借高效的执行性能、严谨的内存控制,长期占据系统开发、底层架构、高性能服务等核心场景。C、C++ 作为经典老牌编译语言,奠定了现代编程的基础;Go、Rust 则作为后起之秀,针对新时代开发痛点(如并发安全、内存安全)进行了革新性设计。本文将从语言定位、核心特性、性能效率、内存管理、并发模型、生态场景等核心维度,对这四大主流编译语言进行全方位对比,帮你清晰认知各语言的优势与适用场景,为技术选型提供参考。
Learn and share.

在编程领域,编译语言凭借高效的执行性能、严谨的内存控制,长期占据系统开发、底层架构、高性能服务等核心场景。C、C++ 作为经典老牌编译语言,奠定了现代编程的基础;Go、Rust 则作为后起之秀,针对新时代开发痛点(如并发安全、内存安全)进行了革新性设计。本文将从语言定位、核心特性、性能效率、内存管理、并发模型、生态场景等核心维度,对这四大主流编译语言进行全方位对比,帮你清晰认知各语言的优势与适用场景,为技术选型提供参考。
以新建文件为例,对比一下几个常见平台的区别。
继续看下MacOS平台的代码:
// os/file.go
// 新建文件
func Create(name string) (*File, error) {
// 跳转到下面的OpenFile
return OpenFile(name, O_RDWR|O_CREATE|O_TRUNC, 0666)
}
// OpenFile在这里还是平台无关的代码
func OpenFile(name string, flag int, perm FileMode) (*File, error) {
testlog.Open(name)
// 从openFileNolog开始,不同平台代码会有不同
f, err := openFileNolog(name, flag, perm)
if err != nil {
return nil, err
}
f.appendMode = flag&O_APPEND != 0
return f, nil
}
// os/file_unix.go
// openFileNolog的unix实现
func openFileNolog(name string, flag int, perm FileMode) (*File, error) {
setSticky := false
if !supportsCreateWithStickyBit && flag&O_CREATE != 0 && perm&ModeSticky != 0 {
if _, err := Stat(name); IsNotExist(err) {
setSticky = true
}
}
var r int
var s poll.SysFile
for {
var e error
//跳转到open
r, s, e = open(name, flag|syscall.O_CLOEXEC, syscallMode(perm))
if e == nil {
break
}
// We have to check EINTR here, per issues 11180 and 39237.
if e == syscall.EINTR {
continue
}
return nil, &PathError{Op: "open", Path: name, Err: e}
}
// open(2) itself won't handle the sticky bit on *BSD and Solaris
if setSticky {
setStickyBit(name)
}
// There's a race here with fork/exec, which we are
// content to live with. See ../syscall/exec_unix.go.
if !supportsCloseOnExec {
syscall.CloseOnExec(r)
}
kind := kindOpenFile
if unix.HasNonblockFlag(flag) {
kind = kindNonBlock
}
// 封装为File结构
f := newFile(r, name, kind)
f.pfd.SysFile = s
return f, nil
}
// os/file_open_unix.go
func open(path string, flag int, perm uint32) (int, poll.SysFile, error) {
// 跳转到syscall.Open
fd, err := syscall.Open(path, flag, perm)
return fd, poll.SysFile{}, err
}
// syscall/zsyscall_darwin_amd64.go
func Open(path string, mode int, perm uint32) (fd int, err error) {
var _p0 *byte
_p0, err = BytePtrFromString(path)
if err != nil {
return
}
// 调用syscall
r0, _, e1 := syscall(abi.FuncPCABI0(libc_open_trampoline), uintptr(unsafe.Pointer(_p0)), uintptr(mode), uintptr(perm))
fd = int(r0)
if e1 != 0 {
err = errnoErr(e1)
}
return
}
// syscall/syscall_darwin.go
func syscall(fn, a1, a2, a3 uintptr) (r1, r2 uintptr, err Errno)
// internal/abi/funcpc.go
func FuncPCABI0(f interface{}) uintptr
// syscall/zsyscall_darwin_amd64.go
func libc_open_trampoline()
//go:cgo_import_dynamic libc_open open "/usr/lib/libSystem.B.dylib"
// 先通过abi.FuncPCABI0(libc_open_trampoline)先获取到open函数的地址
// 然后通过syscall调用open函数
// open函数是libc标准库中的函数,C语言定义为
int open(const char *pathname, int flags, mode_t mode);
//syscall在这里实现
//runtime/sys_darwin.go
//go:linkname syscall_syscall syscall.syscall
//go:nosplit
func syscall_syscall(fn, a1, a2, a3 uintptr) (r1, r2, err uintptr) {
args := struct{ fn, a1, a2, a3, r1, r2, err uintptr }{fn, a1, a2, a3, r1, r2, err}
entersyscall()
//跳转到libcCall
libcCall(unsafe.Pointer(abi.FuncPCABI0(syscall)), unsafe.Pointer(&args))
exitsyscall()
return args.r1, args.r2, args.err
}
func syscall()
// runtime/sys_libc.go
func libcCall(fn, arg unsafe.Pointer) int32 {
// Leave caller's PC/SP/G around for traceback.
gp := getg()
var mp *m
if gp != nil {
mp = gp.m
}
if mp != nil && mp.libcallsp == 0 {
mp.libcallg.set(gp)
mp.libcallpc = getcallerpc()
// sp must be the last, because once async cpu profiler finds
// all three values to be non-zero, it will use them
mp.libcallsp = getcallersp()
} else {
// Make sure we don't reset libcallsp. This makes
// libcCall reentrant; We remember the g/pc/sp for the
// first call on an M, until that libcCall instance
// returns. Reentrance only matters for signals, as
// libc never calls back into Go. The tricky case is
// where we call libcX from an M and record g/pc/sp.
// Before that call returns, a signal arrives on the
// same M and the signal handling code calls another
// libc function. We don't want that second libcCall
// from within the handler to be recorded, and we
// don't want that call's completion to zero
// libcallsp.
// We don't need to set libcall* while we're in a sighandler
// (even if we're not currently in libc) because we block all
// signals while we're handling a signal. That includes the
// profile signal, which is the one that uses the libcall* info.
mp = nil
}
// 跳转到asmcgocall
res := asmcgocall(fn, arg)
if mp != nil {
mp.libcallsp = 0
}
return res
}
// 硬件平台相关代码
// runtime/asm_arm64.s
// func asmcgocall(fn, arg unsafe.Pointer) int32
// Call fn(arg) on the scheduler stack,
// aligned appropriately for the gcc ABI.
// See cgocall.go for more details.
TEXT ·asmcgocall(SB),NOSPLIT,$0-20
MOVD fn+0(FP), R1
MOVD arg+8(FP), R0
MOVD RSP, R2 // save original stack pointer
CBZ g, nosave
MOVD g, R4
// Figure out if we need to switch to m->g0 stack.
// We get called to create new OS threads too, and those
// come in on the m->g0 stack already. Or we might already
// be on the m->gsignal stack.
MOVD g_m(g), R8
MOVD m_gsignal(R8), R3
CMP R3, g
BEQ nosave
MOVD m_g0(R8), R3
CMP R3, g
BEQ nosave
// Switch to system stack.
MOVD R0, R9 // gosave_systemstack_switch<> and save_g might clobber R0
BL gosave_systemstack_switch<>(SB)
MOVD R3, g
BL runtime·save_g(SB)
MOVD (g_sched+gobuf_sp)(g), R0
MOVD R0, RSP
MOVD (g_sched+gobuf_bp)(g), R29
MOVD R9, R0
// Now on a scheduling stack (a pthread-created stack).
// Save room for two of our pointers /*, plus 32 bytes of callee
// save area that lives on the caller stack. */
MOVD RSP, R13
SUB $16, R13
MOVD R13, RSP
MOVD R4, 0(RSP) // save old g on stack
MOVD (g_stack+stack_hi)(R4), R4
SUB R2, R4
MOVD R4, 8(RSP) // save depth in old g stack (can't just save SP, as stack might be copied during a callback)
BL (R1)
MOVD R0, R9
// Restore g, stack pointer. R0 is errno, so don't touch it
MOVD 0(RSP), g
BL runtime·save_g(SB)
MOVD (g_stack+stack_hi)(g), R5
MOVD 8(RSP), R6
SUB R6, R5
MOVD R9, R0
MOVD R5, RSP
MOVW R0, ret+16(FP)
RET
nosave:
// Running on a system stack, perhaps even without a g.
// Having no g can happen during thread creation or thread teardown
// (see needm/dropm on Solaris, for example).
// This code is like the above sequence but without saving/restoring g
// and without worrying about the stack moving out from under us
// (because we're on a system stack, not a goroutine stack).
// The above code could be used directly if already on a system stack,
// but then the only path through this code would be a rare case on Solaris.
// Using this code for all "already on system stack" calls exercises it more,
// which should help keep it correct.
MOVD RSP, R13
SUB $16, R13
MOVD R13, RSP
MOVD $0, R4
MOVD R4, 0(RSP) // Where above code stores g, in case someone looks during debugging.
MOVD R2, 8(RSP) // Save original stack pointer.
BL (R1)
// Restore stack pointer.
MOVD 8(RSP), R2
MOVD R2, RSP
MOVD R0, ret+16(FP)
RET
// 然后回到openFileNolog中
// 在openFileNolog中,继续调用newFile,整体封装为File结构,原路返回
func newFile(fd int, name string, kind newFileKind) *File {
f := &File{&file{
pfd: poll.FD{
Sysfd: fd,
IsStream: true,
ZeroReadIsEOF: true,
},
name: name,
stdoutOrErr: fd == 1 || fd == 2,
}}
pollable := kind == kindOpenFile || kind == kindPipe || kind == kindNonBlock
// If the caller passed a non-blocking filedes (kindNonBlock),
// we assume they know what they are doing so we allow it to be
// used with kqueue.
if kind == kindOpenFile {
switch runtime.GOOS {
case "darwin", "ios", "dragonfly", "freebsd", "netbsd", "openbsd":
var st syscall.Stat_t
err := ignoringEINTR(func() error {
return syscall.Fstat(fd, &st)
})
typ := st.Mode & syscall.S_IFMT
// Don't try to use kqueue with regular files on *BSDs.
// On FreeBSD a regular file is always
// reported as ready for writing.
// On Dragonfly, NetBSD and OpenBSD the fd is signaled
// only once as ready (both read and write).
// Issue 19093.
// Also don't add directories to the netpoller.
if err == nil && (typ == syscall.S_IFREG || typ == syscall.S_IFDIR) {
pollable = false
}
// In addition to the behavior described above for regular files,
// on Darwin, kqueue does not work properly with fifos:
// closing the last writer does not cause a kqueue event
// for any readers. See issue #24164.
if (runtime.GOOS == "darwin" || runtime.GOOS == "ios") && typ == syscall.S_IFIFO {
pollable = false
}
}
}
clearNonBlock := false
if pollable {
if kind == kindNonBlock {
// The descriptor is already in non-blocking mode.
// We only set f.nonblock if we put the file into
// non-blocking mode.
} else if err := syscall.SetNonblock(fd, true); err == nil {
f.nonblock = true
clearNonBlock = true
} else {
pollable = false
}
}
// An error here indicates a failure to register
// with the netpoll system. That can happen for
// a file descriptor that is not supported by
// epoll/kqueue; for example, disk files on
// Linux systems. We assume that any real error
// will show up in later I/O.
// We do restore the blocking behavior if it was set by us.
if pollErr := f.pfd.Init("file", pollable); pollErr != nil && clearNonBlock {
if err := syscall.SetNonblock(fd, false); err == nil {
f.nonblock = false
}
}
runtime.SetFinalizer(f.file, (*file).close)
return f
}
以新建文件为例,对比一下几个常见平台的区别。
继续看下Linux平台的代码:
// os/file.go
// 新建文件
func Create(name string) (*File, error) {
// 跳转到下面的OpenFile
return OpenFile(name, O_RDWR|O_CREATE|O_TRUNC, 0666)
}
// OpenFile在这里还是平台无关的代码
func OpenFile(name string, flag int, perm FileMode) (*File, error) {
testlog.Open(name)
// 从openFileNolog开始,不同平台代码会有不同
f, err := openFileNolog(name, flag, perm)
if err != nil {
return nil, err
}
f.appendMode = flag&O_APPEND != 0
return f, nil
}
// os/file_unix.go
// openFileNolog的unix实现
func openFileNolog(name string, flag int, perm FileMode) (*File, error) {
setSticky := false
if !supportsCreateWithStickyBit && flag&O_CREATE != 0 && perm&ModeSticky != 0 {
if _, err := Stat(name); IsNotExist(err) {
setSticky = true
}
}
var r int
var s poll.SysFile
for {
var e error
//跳转到open
r, s, e = open(name, flag|syscall.O_CLOEXEC, syscallMode(perm))
if e == nil {
break
}
// We have to check EINTR here, per issues 11180 and 39237.
if e == syscall.EINTR {
continue
}
return nil, &PathError{Op: "open", Path: name, Err: e}
}
// open(2) itself won't handle the sticky bit on *BSD and Solaris
if setSticky {
setStickyBit(name)
}
// There's a race here with fork/exec, which we are
// content to live with. See ../syscall/exec_unix.go.
if !supportsCloseOnExec {
syscall.CloseOnExec(r)
}
kind := kindOpenFile
if unix.HasNonblockFlag(flag) {
kind = kindNonBlock
}
// 封装为File结构
f := newFile(r, name, kind)
f.pfd.SysFile = s
return f, nil
}
// os/file_open_unix.go
func open(path string, flag int, perm uint32) (int, poll.SysFile, error) {
// 跳转到syscall.Open
fd, err := syscall.Open(path, flag, perm)
return fd, poll.SysFile{}, err
}
// syscall/syscall_linux.go
func Open(path string, mode int, perm uint32) (fd int, err error) {
// 跳转到openat
return openat(AT_FDCWD, path, mode|O_LARGEFILE, perm)
}
//sys openat(dirfd int, path string, flags int, mode uint32) (fd int, err error)
// syscall/zsyscall_linux_amd64.go
func openat(dirfd int, path string, flags int, mode uint32) (fd int, err error) {
var _p0 *byte
_p0, err = BytePtrFromString(path)
if err != nil {
return
}
// 跳转到Syscall6
r0, _, e1 := Syscall6(SYS_OPENAT, uintptr(dirfd), uintptr(unsafe.Pointer(_p0)), uintptr(flags), uintptr(mode), 0, 0)
fd = int(r0)
if e1 != 0 {
err = errnoErr(e1)
}
return
}
// syscall/syscall_linux.go
func Syscall6(trap, a1, a2, a3, a4, a5, a6 uintptr) (r1, r2 uintptr, err Errno) {
runtime_entersyscall()
// 跳转到RawSyscall6
r1, r2, err = RawSyscall6(trap, a1, a2, a3, a4, a5, a6)
runtime_exitsyscall()
return
}
// N.B. RawSyscall6 is provided via linkname by runtime/internal/syscall.
//
// Errno is uintptr and thus compatible with the runtime/internal/syscall
// definition.
func RawSyscall6(trap, a1, a2, a3, a4, a5, a6 uintptr) (r1, r2 uintptr, err Errno)
// syscall/zsysnum_linux_amd64.go
SYS_OPENAT = 257
// RawSyscall6是通过汇编实现的,传入SYS_OPENAT,最终调用openat函数
// openat函数是libc标准库中的函数,C语言定义为
int openat(int dirfd, const char *pathname, int flags, mode_t mode);
// runtime/internal/syscall/asm_linux_amd64.s // Syscall6 的实现在这里 // func Syscall6(num, a1, a2, a3, a4, a5, a6 uintptr) (r1, r2, errno uintptr) // // We need to convert to the syscall ABI. // // arg | ABIInternal | Syscall // --------------------------- // num | AX | AX // a1 | BX | DI // a2 | CX | SI // a3 | DI | DX // a4 | SI | R10 // a5 | R8 | R8 // a6 | R9 | R9 // // r1 | AX | AX // r2 | BX | DX // err | CX | part of AX // // Note that this differs from "standard" ABI convention, which would pass 4th // arg in CX, not R10. TEXT ·Syscall6<ABIInternal>(SB),NOSPLIT,$0 // a6 already in R9. // a5 already in R8. MOVQ SI, R10 // a4 MOVQ DI, DX // a3 MOVQ CX, SI // a2 MOVQ BX, DI // a1 // num already in AX. SYSCALL CMPQ AX, $0xfffffffffffff001 JLS ok NEGQ AX MOVQ AX, CX // errno MOVQ $-1, AX // r1 MOVQ $0, BX // r2 RET ok: // r1 already in AX. MOVQ DX, BX // r2 MOVQ $0, CX // errno RET
// 然后回到openFileNolog中
// 在openFileNolog中,继续调用newFile,整体封装为File结构,原路返回
func newFile(fd int, name string, kind newFileKind) *File {
f := &File{&file{
pfd: poll.FD{
Sysfd: fd,
IsStream: true,
ZeroReadIsEOF: true,
},
name: name,
stdoutOrErr: fd == 1 || fd == 2,
}}
pollable := kind == kindOpenFile || kind == kindPipe || kind == kindNonBlock
// If the caller passed a non-blocking filedes (kindNonBlock),
// we assume they know what they are doing so we allow it to be
// used with kqueue.
if kind == kindOpenFile {
switch runtime.GOOS {
case "darwin", "ios", "dragonfly", "freebsd", "netbsd", "openbsd":
var st syscall.Stat_t
err := ignoringEINTR(func() error {
return syscall.Fstat(fd, &st)
})
typ := st.Mode & syscall.S_IFMT
// Don't try to use kqueue with regular files on *BSDs.
// On FreeBSD a regular file is always
// reported as ready for writing.
// On Dragonfly, NetBSD and OpenBSD the fd is signaled
// only once as ready (both read and write).
// Issue 19093.
// Also don't add directories to the netpoller.
if err == nil && (typ == syscall.S_IFREG || typ == syscall.S_IFDIR) {
pollable = false
}
// In addition to the behavior described above for regular files,
// on Darwin, kqueue does not work properly with fifos:
// closing the last writer does not cause a kqueue event
// for any readers. See issue #24164.
if (runtime.GOOS == "darwin" || runtime.GOOS == "ios") && typ == syscall.S_IFIFO {
pollable = false
}
}
}
clearNonBlock := false
if pollable {
if kind == kindNonBlock {
// The descriptor is already in non-blocking mode.
// We only set f.nonblock if we put the file into
// non-blocking mode.
} else if err := syscall.SetNonblock(fd, true); err == nil {
f.nonblock = true
clearNonBlock = true
} else {
pollable = false
}
}
// An error here indicates a failure to register
// with the netpoll system. That can happen for
// a file descriptor that is not supported by
// epoll/kqueue; for example, disk files on
// Linux systems. We assume that any real error
// will show up in later I/O.
// We do restore the blocking behavior if it was set by us.
if pollErr := f.pfd.Init("file", pollable); pollErr != nil && clearNonBlock {
if err := syscall.SetNonblock(fd, false); err == nil {
f.nonblock = false
}
}
runtime.SetFinalizer(f.file, (*file).close)
return f
}
最近在读GoLang的源码,源码中有一些跨平台的操作,Go处理的很有意思,在这整理一下。
以新建文件为例,对比一下几个常见平台的区别。
首先看下Windows平台的代码:
// os/file.go
// 新建文件
func Create(name string) (*File, error) {
// 跳转到下面的OpenFile
return OpenFile(name, O_RDWR|O_CREATE|O_TRUNC, 0666)
}
// OpenFile在这里还是平台无关的代码
func OpenFile(name string, flag int, perm FileMode) (*File, error) {
testlog.Open(name)
// 从openFileNolog开始,不同平台代码会有不同
f, err := openFileNolog(name, flag, perm)
if err != nil {
return nil, err
}
f.appendMode = flag&O_APPEND != 0
return f, nil
}
// os/file_windows.go
// openFileNolog的windows实现
func openFileNolog(name string, flag int, perm FileMode) (*File, error) {
if name == "" {
return nil, &PathError{Op: "open", Path: name, Err: syscall.ENOENT}
}
path := fixLongPath(name)
// 跳转到了syscall.Open
r, e := syscall.Open(path, flag|syscall.O_CLOEXEC, syscallMode(perm))
if e != nil {
// We should return EISDIR when we are trying to open a directory with write access.
if e == syscall.ERROR_ACCESS_DENIED && (flag&O_WRONLY != 0 || flag&O_RDWR != 0) {
pathp, e1 := syscall.UTF16PtrFromString(path)
if e1 == nil {
var fa syscall.Win32FileAttributeData
e1 = syscall.GetFileAttributesEx(pathp, syscall.GetFileExInfoStandard, (*byte)(unsafe.Pointer(&fa)))
if e1 == nil && fa.FileAttributes&syscall.FILE_ATTRIBUTE_DIRECTORY != 0 {
e = syscall.EISDIR
}
}
}
return nil, &PathError{Op: "open", Path: name, Err: e}
}
// 封装为File结构
f, e := newFile(r, name, "file"), nil
if e != nil {
return nil, &PathError{Op: "open", Path: name, Err: e}
}
return f, nil
}
// syscall/syscall_windows.go
func Open(path string, mode int, perm uint32) (fd Handle, err error) {
if len(path) == 0 {
return InvalidHandle, ERROR_FILE_NOT_FOUND
}
pathp, err := UTF16PtrFromString(path)
if err != nil {
return InvalidHandle, err
}
var access uint32
switch mode & (O_RDONLY | O_WRONLY | O_RDWR) {
case O_RDONLY:
access = GENERIC_READ
case O_WRONLY:
access = GENERIC_WRITE
case O_RDWR:
access = GENERIC_READ | GENERIC_WRITE
}
if mode&O_CREAT != 0 {
access |= GENERIC_WRITE
}
if mode&O_APPEND != 0 {
access &^= GENERIC_WRITE
access |= FILE_APPEND_DATA
}
sharemode := uint32(FILE_SHARE_READ | FILE_SHARE_WRITE)
var sa *SecurityAttributes
if mode&O_CLOEXEC == 0 {
sa = makeInheritSa()
}
var createmode uint32
switch {
case mode&(O_CREAT|O_EXCL) == (O_CREAT | O_EXCL):
createmode = CREATE_NEW
case mode&(O_CREAT|O_TRUNC) == (O_CREAT | O_TRUNC):
createmode = CREATE_ALWAYS
case mode&O_CREAT == O_CREAT:
createmode = OPEN_ALWAYS
case mode&O_TRUNC == O_TRUNC:
createmode = TRUNCATE_EXISTING
default:
createmode = OPEN_EXISTING
}
var attrs uint32 = FILE_ATTRIBUTE_NORMAL
if perm&S_IWRITE == 0 {
attrs = FILE_ATTRIBUTE_READONLY
if createmode == CREATE_ALWAYS {
// We have been asked to create a read-only file.
// If the file already exists, the semantics of
// the Unix open system call is to preserve the
// existing permissions. If we pass CREATE_ALWAYS
// and FILE_ATTRIBUTE_READONLY to CreateFile,
// and the file already exists, CreateFile will
// change the file permissions.
// Avoid that to preserve the Unix semantics.
h, e := CreateFile(pathp, access, sharemode, sa, TRUNCATE_EXISTING, FILE_ATTRIBUTE_NORMAL, 0)
switch e {
case ERROR_FILE_NOT_FOUND, _ERROR_BAD_NETPATH, ERROR_PATH_NOT_FOUND:
// File does not exist. These are the same
// errors as Errno.Is checks for ErrNotExist.
// Carry on to create the file.
default:
// Success or some different error.
return h, e
}
}
}
if createmode == OPEN_EXISTING && access == GENERIC_READ {
// Necessary for opening directory handles.
attrs |= FILE_FLAG_BACKUP_SEMANTICS
}
if mode&O_SYNC != 0 {
const _FILE_FLAG_WRITE_THROUGH = 0x80000000
attrs |= _FILE_FLAG_WRITE_THROUGH
}
// 跳转CreateFile
return CreateFile(pathp, access, sharemode, sa, createmode, attrs, 0)
}
func CreateFile(name *uint16, access uint32, mode uint32, sa *SecurityAttributes, createmode uint32, attrs uint32, templatefile int32) (handle Handle, err error) {
// 跳转Syscall9
r0, _, e1 := Syscall9(procCreateFileW.Addr(), 7, uintptr(unsafe.Pointer(name)), uintptr(access), uintptr(mode), uintptr(unsafe.Pointer(sa)), uintptr(createmode), uintptr(attrs), uintptr(templatefile), 0, 0)
handle = Handle(r0)
if handle == InvalidHandle {
err = errnoErr(e1)
}
return
}
// syscall/dll_windows.go
// 封装了Syscall9
func Syscall9(trap, nargs, a1, a2, a3, a4, a5, a6, a7, a8, a9 uintptr) (r1, r2 uintptr, err Errno)
// syscall/zsyscall_windows.go
// Syscall9中传入的API名为procCreateFileW
procCreateFileW = modkernel32.NewProc("CreateFileW")
// 实际上最终调用了windows API CreateFileW,下面是CPP版本的API定义
// 到这里,也可以看到,通过Syscall的定义,比较巧妙的做了一定程度上的解耦
HANDLE CreateFileW(
[in] LPCWSTR lpFileName,
[in] DWORD dwDesiredAccess,
[in] DWORD dwShareMode,
[in, optional] LPSECURITY_ATTRIBUTES lpSecurityAttributes,
[in] DWORD dwCreationDisposition,
[in] DWORD dwFlagsAndAttributes,
[in, optional] HANDLE hTemplateFile
);
// runtime/syscall_windows.go
// Syscall9是在这里实现的
//go:linkname syscall_Syscall9 syscall.Syscall9
//go:nosplit
func syscall_Syscall9(fn, nargs, a1, a2, a3, a4, a5, a6, a7, a8, a9 uintptr) (r1, r2, err uintptr) {
return syscall_SyscallN(fn, a1, a2, a3, a4, a5, a6, a7, a8, a9)
}
//go:linkname syscall_SyscallN syscall.SyscallN
//go:nosplit
func syscall_SyscallN(trap uintptr, args ...uintptr) (r1, r2, err uintptr) {
nargs := len(args)
// asmstdcall expects it can access the first 4 arguments
// to load them into registers.
var tmp [4]uintptr
switch {
case nargs < 4:
copy(tmp[:], args)
args = tmp[:]
case nargs > maxArgs:
panic("runtime: SyscallN has too many arguments")
}
lockOSThread()
defer unlockOSThread()
c := &getg().m.syscall
c.fn = trap
c.n = uintptr(nargs)
c.args = uintptr(noescape(unsafe.Pointer(&args[0])))
cgocall(asmstdcallAddr, unsafe.Pointer(c))
return c.r1, c.r2, c.err
}
// 最后,通过cgocall,将go的调用,转换为c的调用
// 然后回到openFileNolog中
// 在openFileNolog中,继续调用newFile,整体封装为File结构,原路返回
func newFile(h syscall.Handle, name string, kind string) *File {
if kind == "file" {
var m uint32
if syscall.GetConsoleMode(h, &m) == nil {
kind = "console"
}
if t, err := syscall.GetFileType(h); err == nil && t == syscall.FILE_TYPE_PIPE {
kind = "pipe"
}
}
f := &File{&file{
pfd: poll.FD{
Sysfd: h,
IsStream: true,
ZeroReadIsEOF: true,
},
name: name,
}}
runtime.SetFinalizer(f.file, (*file).close)
// Ignore initialization errors.
// Assume any problems will show up in later I/O.
f.pfd.Init(kind, false)
return f
}
最近在读go源码。
本来环境都搭建好了,源码也上传git了。
但从另一台电脑下载源码后,报了一堆神奇的错误。
最后发现是go.env文件中,回车换行是按windows系统设定上传到git的,改为linux系统设定就好了。
想起入行以来,因为字符集、换行符、正斜杠反斜杠、tab还是空格,遇到的那堆坑,唏嘘不已。
希望UTF-8早日一统天下,希望各大平台别再特立独行。
非标准化害死人,多套标准更是害死人啊。
最近阅读了Redis6.2源码,添加了一些注释,感兴趣的同学可以看下。
https://github.com/neohope/NeoRedisSrc
| 内核模块 |
内核模块 驱动 |
| 内核对象 |
各类锁【互斥量、信号量、读写锁】 Event【Windows事件对象、Java的wait,可以等待一个或多个】 句柄【进程句柄、对象句柄】 注意只能用于多线程的情况【原子变量、关键区、条件变量是不可以跨进程使用的】 |
| 系统层面 |
系统服务 环境变量 协议调用【URL Scheme】 命令行调用 定时器调用 系统脚本调用 |
| 信号类 |
信号 匿名管道 命名管道 UNIX域 邮件槽 |
| 事件类 |
消息【Windows消息、QT信号槽】 粘贴板 钩子函数 系统内置消息队列 |
| 基于内存 |
共享内存 文件映射 DLL全局数据 DLL注入 |
| 基于文件 |
虚拟文件 实体文件【本地磁盘文件、SAN文件、NAS文件、FTP文件、对象存储文件】 |
| 基于网络 |
Web【WebSocket、SSE、HTTP、REST、SOAP等】 邮件 NetBios函数 数据广播 各类其他Socket通信【TCP、UDP、RAW等】 |
| 基于中间件 |
RPC协议及框架:SAMBA、SOAP、EJB、ICE、Swift、Dubbo等 服务发现:etcd、zk、consol 消息队列【队列、广播、订阅发布】:Kafka、RabbitMQ 数据库:关系【MySQL】、非关系【Redis】 ESB【企业服务总线】 |
| Win系统组件 |
动态数据交换(DDE) OLE技术 COM ALT DCOM |
几个问题
1,不同发行版的docker容器,尤其是glibc这些底层库版本不一致的情况下,可以在同一个宿主下运行,是因为glibc这些库与系统内核提供的ABI一直都保持不变吗?万一内核升级,有ABI变动了,docker是如何处理的呢?
处理办法是镜像自己带着基础库,事实上大多数发行版docker镜像都带着glibc,alpine用的则是更轻量的musl。镜像封装的应用,就只需保证兼容自带的glibc/musl即可。
2,现在虚拟机,都支持将某个虚拟机的窗体,直接投射到宿主机上,让宿主机像操作本地应用来操作虚拟机的应用,这个算那一层的虚拟化呢?也是通过ABI模拟来实现的吗?
说的是vmware unity mode这类功能么?具体如何实现的我没有研究过,无责任猜测只是vmware tool这样的工具提供的屏幕映射。但可以肯定的是,并不影响虚拟化层次,即还是硬件抽象层虚拟化。
3,在windows的vmware或virtualbox运行macos的虚拟机超级慢,但运行ubuntu就还蛮快的,是因为mac到win的ABI很难模拟吗?
“超级缓慢”主要是GUI慢,这个与显示方式和驱动都有关。如果你在SSH到虚拟的ubuntu和macos上,跑个console下的跑分软件的话,得到的分数相差并不大。
Windows 系统中是否有文件、访问、资源的隔离手段?是否存在 Windows 版本的容器运行时呢?
Windows的容器还是太重了,镜像超级大,而且优先支持微软自己的技术栈。这么重,加之不成熟,现在对上服务器虚拟化的各类成熟方案,实在没啥优势可言。
WindowsServer从设计上来讲,模块化做的比linux要差一些,内核做的事情也太多,早期类库变化也过于剧烈,类库前后兼容性也不算好。加之之前主推的技术,生命周期都太短,自家技术代与技术代之间没有传承,闭源得不到社区支持,开源服务端软件,在linux上性能,往往吊打windows。
而且即使是windows的虚拟化软件,微软自家做的也不是最好的。vmware和virtualbox多香。
微软唯一让我感到惊艳了一次的,还是wsl,但总归功能差了些,性能差了些。我倒是觉得,把wsl做好,可以同时支持win和linux容器,windows可能更有机会一些。
再后面,就是微软家生态的事情咯。其实微软做的很多理念,都挺超前的,而且也开始与社区合作。但微软,啥都要自己造轮子,技术延续性差又不断要大家换轮子,又没有强援,做生态挺难的。
看下linux和java,一个轮子能用多少年。看一下苹果,也是闭源,也是自己造轮子,还要交保护费,但苹果也不要你三五年就把轮子换了啊,而且保护费不是白交啊。远了远了。。。
docker
我觉得docker和其他虚拟化技术比,仍是最成功的,从docker到containerd,是一种技术上的成功,但是商业上的失败。
k8s应当与swarm对比。k8s成功,源自于google需要这样的一套工具,用来管理自己的容器,开发来先自己用,积累了很多经验。swarm则没有这个过程。云厂商们,自然选择对自己更好用,更有利的工具咯。
google至此,已占有web流量制高点,浏览器制高点,移动操作系统制高点,云原生制高点。厉害
网络虚拟化
N年前,在大学用虚拟机的时候,就遇到了HostOnly,Bridge,NAT等联网模式,当时对Bridge和NAT的区别也是搞了挺久才弄清楚的。
系统对既有旧方案和旧功能的兼容
这种例子蛮多的,比如:
1、系统升级时,如果API版本升级到了2.0,1.0版本也要长期保留。
2、界面风格变化了,还能允许用户切换回之前的风格
3、做系统替换时,一些用户用惯了的工具、报表什么的,经常会被要求在新系统上增加对应功能
4、系统升级时,尽量去兼容原有数据库设计,而不是推翻重来
5、对于部分政企用户,被迫去兼容IE,全是泪啊
存储
存储小文件:用过本地存储、SSD、SAN、NAS、SFTP、Ceph【对象存储】,云存储【云盘、对象存储】
存储大文件:用过SAN、NAS、Ceph【块存储】、HDFS
存储XML用过existdb、Oracle XML DB
存储JSON用过mongodb、es
备份数据,用过磁带和蓝光盘
试过IPFS,但并没有实际投入使用
从底层介质来看:可以分为SSD、磁盘、磁盘阵列、磁带、光盘等
块存储,相当于划分了一块存储空间或一块逻辑盘,给了操作系统
文件存储,相当于操作系统在磁盘上创建了文件系统,可以作为本地磁盘使用,加上网络访问功能,可以封装成为NAS、SFTP等
HDFS,可以看作分布式的文件存储
对象存储,可以看作分布式的NAS
IPFS,可以看作把一个BT网络,封装成了一个NAS
在我们行业里,使用云存储最大的障碍是两个:
1、使用公有云,担心数据安全、用户隐私的问题;少量上云的数据,也因种种限制,只能用移动联通电信的云;
2、使用私有云,很多机构不愿花大价钱购买服务,又没有能力自己运维,最终很难推进;
但总的来讲,这几年,云存储的使用面还是越来越广的,需要有个渐进的过程。
网络透明
服务网格可以实现透明,很大程度上是服务网格的整个网络环境是相对可控的。
远程通讯,如果在网络可控的环境下,其实完全可以和服务网格采用同样的方式。
但在互联网环境下,无法实现网络的可控,运维工程师、网络工程师是无法把程序员的大部分工作都做掉的,也就是程序员不能只关心数据,不关心网络。
何时能实现这种透明呢?个人认为,需要网络设备更新换代才行,要华为、思科支持这种透明,并能将透明能力,一定程度上开放给开发应用的厂商,才有可能实现一定程度上的透明。
SDN
个人感觉,SDN与服务网格感觉从思想上很相似,实现途径却不一样。
SDN本身十分依赖于网络提供商,其实按我理解,需要一次网络设备更新换代才行,而且网络提供商也需要进行一次大升级,用户也要跟着改造,这一方面需要技术沉淀,另一方面需要很多资金投入,一方面需要等用户升级。SDN概念也比较早,在网络提供商向用户推销SDN概念时,多数网络用户第一反映很可能是一脸懵逼。很多用户根本听不明白,有啥用,现在感觉挺好的啊。
而服务网格,是在可控网络下进行的,不需要网络设备的更替,也不需要以来网络供应商,更不存在跨越网络供应商的问题。大厂支持,再加上K8S和Istio的加持,所以生态就起来了,有滋有味的。服务网格用户是技术人员,服务网格出来推广的时候,微服务已大行其道了。技术人员们被服务治理烦的不要不要的,一听有方法直接把自己解放出来,学习应用热情高涨。加上并不需要夸张的资金投入,方案也就更容易落地了。
共识机制
Basic Paxos在全部节点可信任的时候,主要还是效率问题。所以zk、etcd都要用改进的算法。
Basic Paxos在部分节点不可信任的时候,是不适用的。所以公共区块链项目需要用其他的共识机制。
公共区块链中,常用的共识方法有PoW、PoS、DPoS等,这些方法,一方面是要达成共识进行记账,另一方面是要防止网络被恶意攻击;
联盟链中,一般采用PBFT、PoET等方法,由于不需要面向整个互联网,所以共识机制效率比公共链高不少;同理RAFT、ZAB,默认所有节点是可信任的,效率也是比较高的。
其实大家在日常生活中,默认会通过中心化的方式来达成共识,比如转账成功与否,大家一般会通过银行或支付宝的记录来判断,而不是线下协商。而这种方式,反而是交易效率最高的。
还是那句话,根据业务场景,选择足够用的架构和算法就好了。适合团队,能高效低成本解决问题的方案,就是最好的方案,哪怕这个技术方十分十分的朴素。
为何有了 DNS,还会出现 Eureka、Consul 这些专有的服务发现框架?
感觉其实要解决的问题并不一样:
1、基于DNS,其实要解决的问题是流量的流向,可以做到流量监控,但无法管理到具体业务,功能简单,效率更高;正好符合K8S的需要;
2、服务发现框架,其实是深入到业务层面,在应用框架集成、业务监听支持、健康检查、服务保护等功能。而且基于业务需求,进一步提供配置管理、环境管理等、多数据中心管理、业务紧密结合的功能;
有点像前面说的四层负载和七层负载的意思。
一个功能简单,效率高花头少;一个功能复杂,效率低可发挥空间更大;
那么除了 BFF 之外,你还用网关来做什么?
在API网关之前,会把移动APP的后端服务单独独立出来,APP访问APP后端服务,APP后端服务访问Web后端服务,相当于做了一个完全定制化的API网关。
当时也尝试过直接用Nginx来完成,但当时也不知道OpenResty这类扩展,复杂些的功能没能实现,没能实际应用。
后面从Zuul开始就不造轮子了。
客户端的负载均衡
客户端的负载均衡,在传统的CS解决方案中也有涉及,比如:
1、直连数据库时,统计类SQL直连只读库就好了
2、而且有些其实也变相做了优化,比如大家都熟悉的游戏开新服,登陆时就把流量分了
客户端负载均衡方案,其实和nginx的均衡策略基本一致的。但还遇到过一种中间情况,就是多了一个调度器,客户端定期获取服务IP列表,也可以从调度器获知每个IP的繁忙程度,然后决定要访问那个IP。
此外,云厂商对于移动、联通、电信的网络,有时也会用不同的链路,并会交一定的“保护费”啦。
负载均衡的设计,最佳人选,应该是懂业务的架构师(默认架构懂开发、懂运维)。
但同时,开发、运维团队要参与及评审:
1、比如开发要与架构沟通,团队熟悉哪些技术栈,哪些服务有状态哪些服务无状态等,中间件如何取舍
2、比如运维要与架构沟通,跨机房数据同步要如何做,容灾备份要如何处理,动态扩展是靠脚本还是靠云平台机制等