moved to blog.vaibhavmishra.com

Adios Java , Adios Erlang

| Thursday, April 15, 2010
Don't get it wrong from the title, I did not actually ditched java but turns out I have to look elsewhere , because Java simply do not cut this time, let me walk through the situation, Java is a great language , personally I prefer python and bash for scripting and none of my script goes more than 20 lines so it is not really a benchmark but recently I got a chance to modify a script written in Java, or must I say optimize it, I did my homework and look for several approaches, since it is a thread based application some of the solution I found is
  1. Use real time system - this does not work out because Ubuntu 10.04 real time kernel doesnot play nice and I am too egoist to go for previous version.
  2. Increase hardware - since my real options are only in the clouds hnce effectively VM which generally zeroed the effect of increasing hardware so this also fell off.
  3. Use some real skills -- I did some digging around and increase the performance of script by remove just 2 characters, and that did it, it was flying high in the test environment and I thought I finished or did I...
There lies a twist , it turns out Java Have a limit on number of threads it can create, before running out of memory or some other bullshit limit and believe me I tried really hard to figure ut optimization , even remove all the code from the thread and simply putting it to sleep, but I hit the wall, 5000 threads, 5092 to be precise.
and in production my script fails.

so the obvious solution was to run script simultaneously in several JVMs which would require several machines or at least one super- machine , or change the language, and believe me I didn't like this, but this was a special case , so I turned to option of using other language , few options are

  • C or NPLT which is essentially C-- this was good, really awesome , great performance promises like spawning 100000 threads in 2s ,but really it is C, memory management etc., so I put it as a last resort.
  • erlang - This comes out to be seemingly more viable solution , since it is more high level, a language of new paradigm so perfect for learning and also tailor made for concurrency.
naturally Erlang is the choice , but it seems to have a huge learning curve, everything is different but still , it seems far more elegant than C, and promises to create greater than 32000 threads in action with no seemingly achievable upper limit. let's hope this goes well.

47 comments:

Andrea said...

What about the conclusion that an application that spawn 5000+ concurrent threads has some fundamental problem? (and it isn't the language)

grooveek said...

I agree Java has some flaws, as any other programming language, but where did you see such a limitation (on number of threads created) existed in specs ?
Did you try increasing the file handle limit on your ubuntu box ?

My 2 cents

Cheers

grooveek

Robert Virding said...

Why should spawning 5000+ threads indicate some fundamental problem with an application? In a large inherently concurrent application you can easily get 10000s of processes, over 100000. It is for that type of problem Erlang was designed.

Mike Funk said...

Have you not heard of ThreadPoolExecutors, Callables and Futures in the java.util.concurrent package?

Greg said...

Callables and Futures, while convenient, can often be too costly to use. Having a fixed number of threads pulling data of off specialized queues has proven to be far more performant in my experience.
http://en.wikipedia.org/wiki/Staged_event-driven_architecture
http://jcyclone.sourceforge.net/about.html

Khalil said...

Thread limit on JDK 1.6 seems to be around 15K see http://paultyma.blogspot.com/2008/03/writing-java-multithreaded-servers.html
Would you need that much threads how about using a thread pool, you could also try the new concurrency stuff aka Parallel array http://www.ibm.com/developerworks/java/library/j-jtp11137.html

wekempf said...

Oh, where to begin. If it's in Java, it's not a "script". Any application that used that many threads would never, EVER be run on any computer I ever own, and you couldn't pay me enough to work with the code. This is bad architecture at it's finest. I also wouldn't be surprised if this upper limit you hit was an OS limitation, as it's simply unreasonable. Then there's NPTL (you got the acronym wrong), which is the newer POSIX threading library for Linux. This isn't C (the library is written in C, but that's not relevant to what we're discussing), but a system library. Java can run under NPTL (the Wikipedia page you linked about NPTL states this). So if NPTL could fix your thread limit problem (doubtful), then it probably doesn't require you to rewrite your Java application.

5000 threads? (shudder)

Mats Henricson said...

Migrate to Scala and Akka instead, with its actors. You can have millions of them at the same time.

vinu said...

@andrea actually application is not spawning 5000 threads, I require it to spawn more that that in practice, and I tried to test it in my machine which shows a limit of 5092 threads, here only I used a threads which simply sleeps to test the limit , so it's not the application which have some problem

@groveek this limit is not in specs , I tested it in my machine, I increased hardware by a factor of 2 , then it goes to nearly 8000 threads, after which I had no other hardware to test, I tried this in windows, and the 2nd one in Linux , red hat to be precise, and as per the statement that there is some fundamental flaw with the application , I am not sure, but I need to increase this limit and I think if I have option available(erlang or NPLT) then I should do it though I am back to Ubuntu.
@Mike java.util.Concurrent did not solve my problem, but in fact complicates things more.
@wekemf I agree that if it's java then it's not a script, but my application behaves essentially as script so I referred it to as script,
I would investigate if NPTL would work with java, thanks for this information.

grooveek said...

If you look into HotSpot sources, you'll see that NPTL is handled transparently on Linux distrib that handles it.
NPTL is now part of the glibc since a while and ubuntu should use it, I think
You can see that by doing a little C programs that prints
sysconf(_SC_THREAD_THREADS_MAX)

If this function returns -1, then NPTL will be used by Java HotSpot

That does return -1 on my debian lenny, for example

I post the relevant part of the hotspot code :

n = confstr(_CS_GNU_LIBPTHREAD_VERSION, NULL, 0);
if (n > 0) {
char *str = (char *)malloc(n);
confstr(_CS_GNU_LIBPTHREAD_VERSION, str, n);
// Vanilla RH-9 (glibc 2.3.2) has a bug that confstr() always tells
// us "NPTL-0.29" even we are running with LinuxThreads. Check if this
// is the case. LinuxThreads has a hard limit on max number of threads.
// So sysconf(_SC_THREAD_THREADS_MAX) will return a positive value.
// On the other hand, NPTL does not have such a limit, sysconf()
// will return -1 and errno is not changed. Check if it is really NPTL.
if (strcmp(os::Linux::glibc_version(), "glibc 2.3.2") == 0 &&
strstr(str, "NPTL") &&
sysconf(_SC_THREAD_THREADS_MAX) > 0) {
free(str);
os::Linux::set_libpthread_version("linuxthreads");
} else {
os::Linux::set_libpthread_version(str);
}
} else {
// glibc before 2.3.2 only has LinuxThreads.
os::Linux::set_libpthread_version("linuxthreads");
}

Check on your box to see which version is included

Cheers

Grooveek

DSmith said...

@wekempf
quote: 5000 threads? (shudder)

If we were talking about traditional concurrency models using OS threads, and shared memory, and locks, I would agree; shudder.

Happily there are other options available today, exemplified best by the Erlang concurrency model.

Purohit D said...

Java development services with dedicated Java developers. Java development outsourcing with proficient Java Developers.

香君 said...

rain before seven; fine before eleven.......................................................

BurtonClary031 said...

若對自己誠實,日積月累,就無法對別人不忠了。........................................

珮均 said...

永遠不要躊躇伸出你的手。也永遠不要躊躇接受別人伸出的手。..................................................

姿婷 said...

廢話不多,祝你順心~^^........................................

健豪 said...

完成一個小目標,會把自己推向一個大目標 ....................................................

M12aeganT_Moe12 said...

好文!值得一推~~加油哦!.............................................

建霖 said...

感謝予我如此動感的blog!.........................

730A_ngelinaRabideau0 said...

面對光明,陰影就在我們身後!加油哦! ....................................................

又陽 said...

靠山山倒,靠人人老,靠自己最好。 ............................................................

rlkris said...

色情動畫影成人色情動畫色情介紹男同志色情片男同志色情影片免費男同志網免費性圖片免費東洋影片免費的av片免費的片免費的色情卡通免費直撥網免費直撥影片免費直播短片免費長片分享區免費咪咪影片網免費是訊免費洪爺色情免費洪爺色情網免費性愛觀看影片免費性愛影片分享區免費性愛動畫免費男做愛影片免費男影片免費男優免費亞洲影片免費性小說免費性片下載免費性交貼圖免費性貼片辦公室偷情遊戲免費視訊聊天網交友戀愛進行室嘟嘟成人網

冠儒俊鑫 said...

Many a little makes a mickle...................................................................

淑慧 said...

真是太有道理了~~我支持你~~~.................................................................

黃佳伸 said...

很期待新的內容,支持你.................................................................

江婷 said...

Poverty tries friends..................................................................                           

亦妮亦妮 said...

閒來無聊逛逛blog~~跟您打聲招呼~~.................................................................

彥安彥安 said...

人不能像動物一樣活著,而應該追求知識和美德............................................................

吳婷婷 said...

當一個人內心能容納兩樣相互衝突的東西,這個人便開始變得有價值了。............................................................

慧杰慧杰 said...

第一次來這裡 愛上你的部落格 感謝你的分享............................................................

維哲維哲 said...

累了嗎?來杯咖啡休息一下吧!............................................................

姿柯瑩柯dgdd憶曾g智曾 said...

第一次來這裡 愛上你的部落格 感謝你的分享............................................................

蔡靜芳蔡靜芳 said...

雖天地之大,萬物之多,而惟吾蜩翼之知。.......................................................

潘凱花潘凱花 said...

人生都有不如意的時候~ 但你的BLOG帶給大家愉快的心情..................................................................

楊儀卉 said...

開懷幸福的生活,是每個人的夢想~~希望大家都能夠實現!...............................................................

童祖如童祖如 said...

不論做什麼事,相信自己,別讓別人的一句話,把你擊倒。..................................................

毛彥宇毛彥宇 said...

河水永遠是相同的,可是每一剎那又都是新的。..................................................

凱v胡倫 said...

你不能決定生命的長度,但你可以控制它的寬度..................................................................

家唐銘 said...

當最困難的時候,也就是離成功不遠的時候。..................................................

小杰 said...

愛情不是慈善事業,不能隨便施捨。......................................................................

x于珊姚于珊姚于珊 said...

財富並非永遠的朋友,但朋友卻是永遠的財富。......................................................................

怡禹玄禹玄君 said...

不會從失敗中找尋教訓的人,成功之路是遙遠的。.................................................

翊翊翊翊張瑜翊翊翊 said...

第一次睇你blog,鐘意!............................................................

麗王王珠 said...

寂寞又無聊 看到你的BLOG 加油喔!!............................................................

立和辛和胡辛和辛偉 said...

單純喜歡你的部落格,希望你能收到我的感謝 ^^..................................................................

黃英吳思潔吳思潔邦 said...

嗯~蠻不錯耶~~我喜歡 ∩ 3∩............................................................

metallic said...

Java has some flaws but
java has few real concept that made internet more wide than before Java applet is such an example.
Any way i found another blog that post real about programming.
Great job.