|
马上注册,结交更多好友,享用更多功能,让你轻松玩转社区。
您需要 登录 才可以下载或查看,没有账号?注册
×
QUOTE
Linux BGP-router perfomancePosted by Carrie on Friday, May 6, 2005 - 10:09 (Sorry for my English)Problem summary: How to improve linux router perfomance which has very very large routing table and high traffic?Description:I work in small ISP which uses usual Linux router as BGP router and everything is ok.But in last monthes I noticed that soon router will not be able to do its work fine because of growing traffic. And I began to do some tests to find out what maximum perfomace can be achieved by linux router.Test setup:host1 (linux) | eth0 ------- eth0 | router (linux) | eth1 ------- eth0 | host2 (linux)host1: eth0 - 10.0.0.2/24 (will receive packets from host2)router: eth0 - 10.0.0.1/24eth1 - 10.0.1.2/24host2: eth0 - 10.0.1.1/24 (will generate packets with pktgen)Router is a new brand dual opteron246 (2Ghz) with 1Gb ram and Intel dual port 1000MT server adapter (e1000). Linux distribution: debian sarge (kernel 2.4.8-686-smp, 2.4.11-686-smp, NAPI is turned off), CentOS4 (RHEL4 clone, kernel-2.6.9-amd64-smp, NAPI is turned on). Test1 (simple):host2 generates 140k packets per second to host1 during 24hours.Result: all packets was received by host1, router average idle - 71%(centos). Thus there are no problem for linux to route such traffic but it seems limit exists. And I didn't try to find out perfomance limits in this test: it isn't realistic.Test2:To perform more realistic tests I took routing table from my real BGP router and copied it to test router with one modification: I replaced gateway of each entry to 10.0.0.2 (host1 ip). And there was about 160k entries in routing table (we have 2 higher ISPs and each announces full-view).Then I found that there was about 150 entries for network 66.166.0.0/15 in routing table and for this once host2 generated 140k packets per second to 66.166.0.0/15 during 24hours.Result: all packets was received by host1, router average idle - 55% (centos, debian).Test2.2:Same as test2 but Router is FreeBSD-5.3.Result: all packets was received by host1, router average idle - 58%.Test3:Same as test2 but with ip_conntrack loaded (modprobe ip_conntrack hashsize=100003) and new configuration for pktgen:......pgset "dst_min 66.166.0.0"pgset "dst_max 66.167.255.255"pgset "flag IPDST_RND"pgset "flag UDPDST_RND"pgset "flag UDPSRC_RND"pgset "udp_src_min 1024"pgset "udp_src_max 1033"pgset "udp_dst_min 1024"pgset "udp_dst_max 1033"......Result: about 200 packets was lost by router ("ip_conntrack: table full, dropping packet"), router average idle - 47% (centos, debian).NOTE: I don't need such large ip_conntrack hash actually. This only shows me perfomance drop.Test4:Same as test3 but ip_conntrack hashsize=362879.Results:- For CentOS: about 20 packets was lost by router ("ip_conntrack: table full, dropping packet"), some kernel warnings ("Losing some ticks... checking if CPU frequency changed.", "Your time source seems to be instable or some driver is hogging interupts"), router average idle - 35%;- For Debian: there was many kernel errors ("kswapd0: page allocation failure") and it hanged up.TODO: Test5: To measure jitter.PROBLEM: I have maximum about 50k packets per second (pps) on my real BGP router at this time. And I think 100-150k pps is not enough for me to forget about this problem for 1-2 years at least.What can I do if I want to stay with linux?Thank you for any suggestion. |
|