服务器开发(Server Development)

Linux copyfile code

Fri, 14 Nov 2008 04:00:29 +0000

Linux下面如何复制文件的函数代码：

[codes=c]

#include
#include
#include

using namespace std;

bool copyfile(const string& srcfile, const string& destfile)
{
        int        fdin, fdout;
        void        *src, *dst;
        struct stat statbuf;

        if ((fdin = open(srcfile.c_str(), O_RDONLY)) < 0)
            return false;
        if ((fdout = open(destfile.c_str(), O_RDWR | O_CREAT | O_TRUNC,
          FILE_MODE)) < 0)
            return false;
        if (fstat(fdin, &statbuf) < 0)  /* need size of input file */
            return false;
        /* set size of output file */
        if (lseek(fdout, statbuf.st_size - 1, SEEK_SET) == -1)
            return false;
        if (write(fdout, "", 1) != 1)
            return false;

        if ((src = mmap(0, statbuf.st_size, PROT_READ, MAP_SHARED,
          fdin, 0)) == MAP_FAILED)
            return false;
        if ((dst = mmap(0, statbuf.st_size, PROT_READ | PROT_WRITE,
          MAP_SHARED, fdout, 0)) == MAP_FAILED)
            return false;

        memcpy(dst, src, statbuf.st_size); /* does the file copy */
        return true;
}
[/codes]
参考：
http://topic.csdn.net/u/20071109/17/ad236653-c55c-4c09-bb27-1b7f14c75af1.html
Tags - linux

出差北京

Fri, 14 Nov 2008 03:43:19 +0000

希望一切顺利！
Cancan乖！
团队队员们辛苦了！
Eiji,SW,Jack,WZX,ZK,LB.努力！
Hansi努力！

祝福一下Michael

Sat, 08 Nov 2008 07:41:08 +0000

在一次偶然的机会，去了Michael的博客，并交换了连接。之后偶尔交流请教一下，也知道他是六间房的创始人之一，技术和为人都非常不错，而且尤其称道的是他的台球，并有点印象可乐吧和他有些渊源。这个地方曾经自己也去玩过，虽然玩的不好。

后来在一次做图片分离服务器设计的时候，请教了tu.6.cn的架构思路，很感激Michael的指点。

最近的几天，关注Michael的博客的时候，了解到Michael的一些近况，感觉不甚公平，当然回想一下，或许并不像自己想象的那么坏。Michael依然是Michael，互联网的不景气会给很多行业带来不景气，但是从很多角度来说，依然会有很多的商机，如果可以把握好的话，或许是更好的创业机会，而希望的是Michael能够重新抖擞精神，做回原来的自己。

寥寥数语，仅表祝福之心。
Tags - michael

抛弃IE,投入Firefox怀抱

Sat, 08 Nov 2008 07:32:19 +0000

IE的不爽终于让我忍无可忍，决定彻底放弃之，才发现Firefox已经增加很多优秀的功能，速度也比之前有所提高，现在来看有一点不足就是打开多个页面的时候，如果想关闭之前的某个页面，会一直无法关闭，直到该页面加载完毕。

Tags - ie , firefox

域名指向中...

Wed, 05 Nov 2008 06:05:42 +0000

由于备案原因，只得暂时指向libing.net.cn域名，新的域名备案下来将继续使用！谢谢大家关注。

谴责Baidu行为

Mon, 03 Nov 2008 07:17:05 +0000

已经很少关注baidu关键字排名了，因为自己的关键字一直是比较好！这次给朋友展示的时候，竟然发现自己的关键字被剔除，很是无语。而通过site:libing.net.cn搜索依然有17000多篇文章被收录，就是没有排名，拉开了16页，没有看到一篇相关文章，之前的时候。从我的感觉，可能又是所谓的潜规则了！

除了BS我想没有什么可以多说的！当然，对于大部分的想了解服务器开发的朋友来说，我觉得是一种损失，这种损失是由我们光荣的名族企业Baidu。

或许我说的有点大，只是希望多一些公平，少一些潜规则。
Tags - baidu

Simpler Multithreading in C++0x

Fri, 31 Oct 2008 08:09:42 +0000

原文地址：http://www.devx.com/SpecialReports/Article/38883
//
One major new feature in the C++0x standard is multi-threading support. Prior to C++0x, any multi-threading support in your C++ compiler has been provided as an extension to the C++ standard, which has meant that the details of that support varies between compilers and platforms. However, with the new standard, all compilers will have to conform to the same memory model and provide the same facilities for multi-threading (though implementors are still free to provide additional extensions). What does this mean for you? It means you'll be able to port multi-threaded code between compilers and platforms with much reduced cost. This will also reduce the number of different APIs and syntaxes you’ll have to know when writing for multiple platforms.
The core of the new thread library is the std::thread class, which manages a thread of execution, so let's start by looking at that.

Launching Threads
You start a new thread by constructing an instance of std::thread with a function. This function is then used as the entry point for the new thread, and once that function returns, the thread is finished:

[codes=c]
    void do_work();
    std::thread t(do_work);
[/codes]

This is just like the thread-creation APIs we're all used to—but there's a crucial difference: This is C++, so we're not restricted to functions. Just like many of the algorithms in the Standard C++ Library, std::thread will accept an object of a type that implements the function call operator (operator()), as well as ordinary functions:
[codes=c]
    class do_work
    {
    public:
        void operator()();
    };

    do_work dw;
    std::thread t(dw);
[/codes]
It's important to note that this actually copies the supplied object into the thread. If you really want to use the object you supplied (in which case, you'd better make sure that it doesn't get destroyed before the thread finishes), you can do so by wrapping it in std::ref:
[codes=c]
    do_work dw;
    std::thread t(std::ref(dw));
[/codes]
Most thread creation APIs allow you to pass a single parameter to your newly created thread, typically a long or a void*. std::thread allows arguments too, but you can pass any number, of (almost) any type. Yes, you read that right: any number of arguments. The constructor uses C++0x's new variadic template facility to allow a variable number of arguments like the old ... varargs syntax, but in a type-safe manner.
You can now pass objects of any copyable type as arguments to the thread function:

[codes=c]
    void do_more_work(int i,std::string s,std::vector v);
    std::thread
        t(do_more_work,42,"hello",std::vector(23,3.141));
[/codes]
Just as with the function object itself, the arguments are copied into the thread before the function is invoked, so if you want to pass a reference you need to wrap the argument in std::ref:
[codes=c]
    void foo(std::string&);
    std::string s;
    std::thread t(foo,std::ref(s));
[/codes]
OK, that's enough about launching threads. What about waiting for the thread to finish? The C++ Standard calls that "joining" with the thread (after the POSIX terminology), and you do that with the join() member function:
[codes=c]
    void do_work();
    std::thread t(do_work);
    t.join();
[/codes]
If you're not planning on joining with your thread, just destroy the thread object or call detach():
[codes=c]
    void do_work();
    std::thread t(do_work);
    t.detach();
[/codes]
Now, it's very well launching all these threads, but if you're going to share data you'd better protect it. The new C++ Standard Library provides facilities for that, too.

Protecting Data
In the C++0x thread library, as with most thread APIs, the basic facility for protecting shared data is the mutex. In C++0x, there are four varieties of mutexes:

引用

non-recursive (std::mutex)
recursive (std::recursive_mutex)
non-recursive that allows timeouts on the lock functions (std::timed_mutex)
recursive mutex that allows timeouts on the lock functions (std::recursive_timed_mutex)

All of them provide exclusive ownership for one thread. If you try and lock a non-recursive mutex twice from the same thread without unlocking in between, you get undefined behavior. A recursive mutex simply increases the lock count—you must unlock the mutex the same number of times that you locked it—in order for other threads to be allowed to lock the mutex.

Though these mutex types all have member functions for locking and unlocking, in most scenarios the best way to do it is with the lock class templates std::unique_lock<> and std::lock_guard<>. These classes lock the mutex in the constructor and release it in the destructor. Thus, if you use them as local variables, your mutex is automatically unlocked when you exit the scope:

[codes=c]
    std::mutex m;
    my_class data;

    void foo()
    {
        std::lock_guard lk(m);
        process(data);
    }   // mutex unlocked here
[/codes]
std::lock_guard is deliberately basic and can only be used as shown. On the other hand, std::unique_lock allows for deferred locking, trying to lock, trying to lock with a timeout, and unlocking before the object is destroyed. If you've chosen to use std::timed_mutex because you want the timeout on the locks, you probably need to use std::unique_lock:
[codes=c]
    std::timed_mutex m;
    my_class data;

    void foo()
    {
        std::unique_lock
            lk(m,std::chrono::milliseconds(3)); // wait up to 3ms
        if(lk) // if we got the lock, access the data
            process(data);
    }   // mutex unlocked here
[/codes]
These lock classes are templates, so they can be used with all the standard mutex types, plus any additional types that supply lock() and unlock() functions.
Protecting Against Deadlock When Locking Multiple Mutexes
Occasionally, an operation requires you to lock more than one mutex. Done wrong, this is a nasty source of deadlocks: Two threads can try and lock the same mutexes in the opposite order, with each end upholding one mutex and waiting for the other thread to finish with the other mutexes. The C++0x thread library allievates this problem, in those cases where you wish to acquire the locks together, by providing a generic std::lock function that can lock multiple mutexes at once. Rather than calling the lock() member function on each mutex in turn, you pass them to std::lock(), which locks them all without risking deadlock. You can even pass in currently unlocked instances of std::unique_lock<>:

[codes=c]
    struct X
    {
        std::mutex m;
        int a;
        std::string b;
    };

    void foo(X& a,X& b)
    {
        std::unique_lock lock_a(a.m,std::defer_lock);
        std::unique_lock lock_b(b.m,std::defer_lock);
        std::lock(lock_a,lock_b);

        // do something with the internals of a and b
    }
[/codes]
In the above example, suppose you didn't use std::lock. This could possibly result in a deadlock if one thread did foo(x,y) and another did foo(y,x) for two X objects x and y. With std::lock, this is safe.

Protecting Data During Initialization
If your data only needs protecting during its initialization, using a mutex is not the answer. Doing so only leads to unnecessary synchronization after initialization is complete. The C++0x standard provides several ways of dealing with this.

First, suppose your constructor is declared with the new constexpr keyword and satisfies the requirements for constant initialization. In this case, an object of static storage duration, initialized with that constructor, is guaranteed to be initialized before any code is run as part of the static initialization phase. This is the option chosen for std::mutex, because it eliminates the possibility of race conditions with initialization of mutexes at a global scope:

[codes=c]
class my_class
    {
        int i;

    public:
        constexpr my_class():i(0){}

        my_class(int i_):i(i_){}

        void do_stuff();
    };

    my_class x; // static initialization with constexpr constructor

    int foo();
    my_class y(42+foo()); // dynamic initialization

    void f()
    {
        y.do_stuff(); // is y initialized?
    }
[/codes]
Your second option is to use a static variable at block scope. In C++0x, initialization of block scope static variables happens the first time the function is called. If a second thread should call the function before the initialization is complete, then that second thread has to wait:
[codes=c]
    void bar()
    {
        static my_class z(42+foo()); // initialization is thread-safe

        z.do_stuff();
    }
[/codes]
If neither options apply (perhaps because the object is dynamically allocated), then it's best to use std::call_once and std::once_flag. As the name suggests, when std::call_once is used in conjunction with a specific instance of type std::once_flag, the specified function is called exactly once:
[codes=c]
    my_class* p=0;
    std::once_flag p_flag;

    void create_instance()
    {
        p=new my_class(42+foo());
    }

    void baz()
    {
        std::call_once(p_flag,create_instance);
        p->do_stuff();
    }
[/codes]
Just as with the std::thread constructor, std::call_once can take function objects instead of functions, and can pass arguments to the function. Again, copying is the default, and you have to use std::ref if you want a reference.

Waiting for Events
If you're sharing data between threads, you often need one thread to wait for another to perform some action, and you want to do this without consuming any CPU time. If a thread is simply waiting for its turn to access some shared data, then a mutex lock can be sufficient. However, generally doing so won't have the desired semantics.

The simplest way to wait is to put the thread to sleep for a short period of time. Then check to see if the desired action has occurred when the thread wakes up. It's important to ensure that the mutex you use to protect the data indicating that the event has occurred is unlocked whilst the thread is sleeping:

[codes=c]
    std::mutex m;
    bool data_ready;

    void process_data();

    void foo()
    {
        std::unique_lock lk(m);
        while(!data_ready)
        {
            lk.unlock();
            std::this_thread::sleep_for(std::chrono::milliseconds(10));
            lk.lock();
        }
        process_data();
    }
[/codes]
This method may be simplest, but it's less than ideal for two reasons. Firstly, on average, the thread will wait five ms (half of ten ms) after the data is ready before it will wake in order to check. This may cause a noticeable lag in some cases. Though this can be improved by reducing the wait time, it exacerbates the second problem: the thread has to wake up, acquire the mutex, and check the flag every ten ms—even if nothing has happened. This consumes CPU time and increases contention on the mutex, and thus potentially slows down the thread performing the task for which it's waiting!
If you find yourself writing code like that, don't: Use condition variables instead. Rather than sleeping for a fixed period, you can let the thread sleep until it has been notified by another thread. This ensures that the latency between being notified and the thread waking is as small as the OS will allow, and effectively reduces the CPU consumption of the waiting thread to zero for the entire time. You can rewrite foo to use a condition variable like this:

[codes=c]
    std::mutex m;
    std::condition_variable cond;

    bool data_ready;

    void process_data();

    void foo()
    {
        std::unique_lock lk(m);
        while(!data_ready)
        {
            cond.wait(lk);
        }
        process_data();
    }
[/codes]
Note that the above code passes in the lock object lk as a parameter to wait(). The condition variable implementation then unlocks the mutex on entry to wait(), and locks it again on exit. This ensures that the protected data can be modified by other threads whilst this thread is waiting. The code that sets the data_ready flag then looks like this:
[codes=c]
    void set_data_ready()
    {
        std::lock_guard lk(m);
        data_ready=true;
        cond.notify_one();
    }
[/codes]
You still need to check that the data is ready though, since condition variables can suffer from what are called spurious wakes: The call to wait() may return even though it wasn't notified by another thread. If you're worried about getting this wrong, you can pass that responsibility off to the standard library too, if you tell it what you're waiting for with a predicate. The new C++0x lambda facility makes this really easy:
[codes=c]
    void foo()
    {
        std::unique_lock lk(m);
        cond.wait(lk,[]{return data_ready;});
        process_data();
    }
[/codes]
What if you don't want to share your data? What if you want exactly the opposite: For each thread to have its own copy? This is the scenario addressed by the new thread_local storage duration keyword.

Thread Local Data
The thread_local keyword can be used with any object declaration at namespace scope at local scope, and specifies that such a variable is thread local. Each thread thus has its own copy of that variable, and that copy exists for the entire duration of that thread. It is essentially a per-thread static variable, so each thread's copy of a variable declared at local scope is initialized the first time that particular thread passes through the declaration, and they retain their values until that thread exits:
[codes=c]
    std::string foo(std::string const& s2)
    {
        thread_local std::string s="hello";

        s+=s2;
        return s;
    }
[/codes]

In this function, each thread's copy of s starts life with the contents "hello." Every time the function is called, the supplied string is appended to that thread's copy of s. As you can see from this example, this even works with class types that have constructors and destructors (such as std::string), which is an improvement over the pre-C++0x compiler extensions.
Thread-local storage isn't the only change to the concurrency support in the core language: There's also a brand new multi-threading aware memory model, with support for atomic operations.

The New Memory Model and Atomic Operations
Sticking to using locks and condition variables to protect your data, you won't need to worry about the memory model. The memory model guarantees to protect your data from race conditions—if you use locks correctly. You'll get undefined behavior if you don't.

If you're working at a really low-level and providing high-performance library facilities, then it's important to know the details—which are too complicated to go into here. For now, it's enough to know that C++0x has a set of atomic types corresponding to the built-in integer types and void pointers—and a template std::atomic<>—which can be used to create an atomic version of a simple user-defined type. You can look up the relevant documentation for the details.

That's All, Folks!
And that’s your whistle-stop tour of the new C++0x threading facilities, which has barely scratched the surface. There's much more to the library, with features such as thread IDs and asynchronous future values.

Tags - c++0x , c++ , multithreading

迟到的C++0x

Fri, 31 Oct 2008 07:59:59 +0000

引用

It's been 10 years since the first ISO C++ standard, and 2009 will bring us the second.

已经10年了，虽然当初的时候并不认识C++.去年的时候已经沸沸扬扬了很久，比如泛型，比如线程支持，还有称将boost加入C++ Standard Library,当然还有GC也据说会得到更好的补充，比如可以开启GC或者关闭GC以达到各种支持。一些新的语言的推出已经抢走了很大的用户群体了。也希望这次的标准可以更大的改变一些。

Tags - c++0x

Boost ASIO 异步TCP ECHO Server[Boost]

Fri, 31 Oct 2008 04:01:00 +0000

本文来自Boost官方网站
http://www.boost.org/doc/libs/1_36_0/doc/html/boost_asio/example/echo/async_tcp_echo_server.cpp

从代码中我们可以看到，首先是session类的定义，session表示一个会话，可以代表某一个客户端socket对象，而server对象表示为所有session提供服务，最后在main里面定义server对象，并传入io_server参数，启动server.

[codes=c]
//
// async_tcp_echo_server.cpp
// ~~~~~~~~~~~~~~~~~~~~~~~~~
//
// Copyright (c) 2003-2008 Christopher M. Kohlhoff (chris at kohlhoff dot com)
//
// Distributed under the Boost Software License, Version 1.0. (See accompanying
// file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
//

#include
#include
//包含Boost 函数绑定
#include
//包含Boost ASIO库头文件
#include

//使用命名空间boost::asio::ip::tcp
using boost::asio::ip::tcp;

//定义Session类表示每个客户端连接信息
class session
{
public:
//构造函数，使用传入的io_service初始化socket_对象
  session(boost::asio::io_service& io_service)
    : socket_(io_service)
  {
  }
//获取socket对象
  tcp::socket& socket()
  {
    return socket_;
  }
//启动处理

  void start()
  {
//调用socket_的异步读取数据函数async_read_some，将数据读入data_数组中
    socket_.async_read_some(boost::asio::buffer(data_, max_length),
//调用boost bind将回调函数绑定为session::handle_read
        boost::bind(&session::handle_read, this,
          boost::asio::placeholders::error,
          boost::asio::placeholders::bytes_transferred));
  }
//处理数据函数handle_read
  void handle_read(const boost::system::error_code& error,
      size_t bytes_transferred)
  {
//如果非error 进行处理
    if (!error)
    {
//异步将data_数组数据，长度为bytes_transferred写入到socket_中，并绑定函数session::handle_write
      boost::asio::async_write(socket_,
          boost::asio::buffer(data_, bytes_transferred),
          boost::bind(&session::handle_write, this,
            boost::asio::placeholders::error));
    }
//否则清除资源
    else
    {
      delete this;
    }
  }
//函数handle_write
  void handle_write(const boost::system::error_code& error)
  {
    if (!error)
    {
//继续读取数据
      socket_.async_read_some(boost::asio::buffer(data_, max_length),
          boost::bind(&session::handle_read, this,
            boost::asio::placeholders::error,
            boost::asio::placeholders::bytes_transferred));
    }
    else
    {
      delete this;
    }
  }

private:
//socket对象
  tcp::socket socket_;
  enum { max_length = 1024 };
//存储数据空间数组
  char data_[max_length];
};

//定义server对象
class server
{
public:
//构造函数  通过传入的io_service初始化内部对象io_service_
//并初始化acceptor_对象
  server(boost::asio::io_service& io_service, short port)
    : io_service_(io_service),
      acceptor_(io_service, tcp::endpoint(tcp::v4(), port))
  {
//定义新的session对象
    session* new_session = new session(io_service_);
//调用acceptor_异步连接该session,并绑定处理accept函数handle_accept
    acceptor_.async_accept(new_session->socket(),
        boost::bind(&server::handle_accept, this, new_session,
          boost::asio::placeholders::error));
  }
//处理accept函数
  void handle_accept(session* new_session,
      const boost::system::error_code& error)
  {
    if (!error)
    {
//调用new_session的启动函数
      new_session->start();
//初始化new_session对象
      new_session = new session(io_service_);
      acceptor_.async_accept(new_session->socket(),
          boost::bind(&server::handle_accept, this, new_session,
            boost::asio::placeholders::error));
    }
    else
    {
      delete new_session;
    }
  }

private:
//定义io_service对象
  boost::asio::io_service& io_service_;
//定义acceptor对象
  tcp::acceptor acceptor_;
};

int main(int argc, char* argv[])
{
  try
  {
    if (argc != 2)
    {
      std::cerr << "Usage: async_tcp_echo_server \n";
      return 1;
    }
//定义io_service
    boost::asio::io_service io_service;

    using namespace std; // For atoi.
//定义server 对象s
    server s(io_service, atoi(argv[1]));
//启动
    io_service.run();
  }
  catch (std::exception& e)
  {
    std::cerr << "Exception: " << e.what() << "\n";
  }

  return 0;
}
[/codes]
Tags - 服务器 , asio

IOCP2.1计划

Fri, 31 Oct 2008 03:30:12 +0000

在文件传输以及某个比较大的Buffer传输的时候，可能会考虑通过压缩方式传递，但是系统API中有TransmitFile and TransmitPackets两个函数，可以起到这个作用。

IOCP2.1计划：
       增加两个功能函数，TransmitFile 和 TransmitPackets。

http://msdn.microsoft.com/zh-cn/magazine/cc302334(en-us).aspx 提到：

Winsock offers two functions for transmitting data that are optimized for file and memory transfers. The TransmitFile API is present on both Windows NT 4.0 and Windows 2000, while TransmitPackets is a new Microsoft extension function that is expected to be available in a future release of Windows. TransmitFile allows the contents of a file to be transferred on a socket. Normally, if an application were to send the contents of a file over a socket, it would have to call CreateFile to open the file and then loop on ReadFile and WSASend until the entire file was read. This is very inefficient because each ReadFile and WSASend call requires a transition from user mode to kernel-mode. TransmitFile simply requires an open handle to the file to transmit and the number of bytes to transfer. The overhead is incurred when opening the file via CreateFile, followed by a single kernel-mode transition. If your app sends the contents of files over sockets, this is the API to use.
      The TransmitPackets API takes the TransmitFile API a step further by allowing the caller to specify multiple file handles and memory buffers to be transmitted in a single call. The function prototype looks like this:
[codes=c]
BOOL TransmitPackets(
  SOCKET hSocket,
  LPTRANSMIT_PACKET_ELEMENT lpPacketArray,
  DWORD nElementCount,
  DWORD nSendSize,
  LPOVERLAPPED lpOverlapped,
  DWORD dwFlags
);
[/codes]
      The lpPacketArray is an array of structures. Each entry can specify either a file handle or a memory buffer to be transmitted. The structure is defined as:
[codes=c]
typedef struct _TRANSMIT_PACKETS_ELEMENT {
    DWORD dwElFlags;
    DWORD cLength;
    union {
        struct {
            LARGE_INTEGER     nFileOffset;
            HANDLE            hFile;
            };
            PVOID             pBuffer;
    };
} TRANSMIT_FILE_BUFFERS;
[/codes]
The fields are self explanatory. The dwElFlags field identifies whether the current element specifies a file handle or memory buffer via the constants TF_ELEMENT_FILE and TF_ELEMENT_MEMORY. The cLength field dictates how many bytes to send from the given data source (a zero indicates the entire file in the case of a file element). The unnamed union then contains the memory buffer of file handle (and possible offset) of the data to be sent.
      Another benefit of using these two APIs is that you can reuse the socket handle by specifying the TF_REUSE_SOCKET flag in addition to the TF_DISCONNECT flag. Once the API completes the data transfer, a transport-level disconnect is initiated. The socket can then be reused in an AcceptEx call. Using this optimization would lessen the overhead associated with creating sockets in the separate accept thread, as discussed earlier.
      The only caveat of using either of these two extension APIs is that on Windows NT Workstation or Windows 2000 Professional only two requests will be processed at a time. You must be running on Windows NT or Windows 2000 Server, Windows 2000 Advanced Server, or Windows 2000 Data Center to get full usage of these specialized APIs.
Tags - 服务器iocp , transmitpackets , transmitfile