Comments on: On bulk loading data into Mnesia http://www.metabrew.com/article/on-bulk-loading-data-into-mnesia/ Erlang, PHP, C, C++, Java, PostgreSQL, MySQL, Hadoop, Linux, awk, bash, sed, grep, screen, vim, irc, ssh etc... Wed, 08 Sep 2010 09:00:01 +0000 hourly 1 http://wordpress.org/?v=3.0 By: A Million-user Comet Application with Mochiweb, Part 3 | Weez.com http://www.metabrew.com/article/on-bulk-loading-data-into-mnesia//comment-page-1#comment-16122 A Million-user Comet Application with Mochiweb, Part 3 | Weez.com Mon, 15 Mar 2010 09:55:47 +0000 http://www.metabrew.com/?p=82#comment-16122 [...] ** Mnesia is overloaded messages you would (probably) otherwise see. Refer to my previous post: On bulk loading data into Mnesia for alternative ways to load in lots of data. The best solution seems to be (as pointed out in the [...] [...] ** Mnesia is overloaded messages you would (probably) otherwise see. Refer to my previous post: On bulk loading data into Mnesia for alternative ways to load in lots of data. The best solution seems to be (as pointed out in the [...]

]]> By: Weclizeclourl http://www.metabrew.com/article/on-bulk-loading-data-into-mnesia//comment-page-1#comment-7561 Weclizeclourl Wed, 05 Aug 2009 15:39:57 +0000 http://www.metabrew.com/?p=82#comment-7561 That's not my fault. --------------------------------------- signature: <a href="http://casualemployment.info/" rel="nofollow">prilosec 20 mg</a> fge55fe9e9e9f8fufjfjfjfffex That’s not my fault.
—————————————
signature: prilosec 20 mg fge55fe9e9e9f8fufjfjfjfffex

]]> By: Per Melin http://www.metabrew.com/article/on-bulk-loading-data-into-mnesia//comment-page-1#comment-3490 Per Melin Tue, 12 May 2009 23:38:35 +0000 http://www.metabrew.com/?p=82#comment-3490 Are you doing all this from the shell? Handling large datasets in the shell has been known to be a bad idea for performance. I tried the ets trickery with 1.5 million records on my MacBook Pro. Took me 30 seconds from the shell and 19 second when I put it in a module (not including the conversion to disc_copies). Are you doing all this from the shell?

Handling large datasets in the shell has been known to be a bad idea for performance.

I tried the ets trickery with 1.5 million records on my MacBook Pro. Took me 30 seconds from the shell and 19 second when I put it in a module (not including the conversion to disc_copies).

]]> By: fred flintstone http://www.metabrew.com/article/on-bulk-loading-data-into-mnesia//comment-page-1#comment-2023 fred flintstone Thu, 02 Apr 2009 17:36:24 +0000 http://www.metabrew.com/?p=82#comment-2023 Maybe it's something to do with table locking (LockKind)? The mnesia reference book mentions this somewhere. Maybe it’s something to do with table locking (LockKind)? The mnesia reference book mentions this somewhere.

]]> By: 用Mochiweb打造百万级Comet应用，第三部分 - IDISC的生活 http://www.metabrew.com/article/on-bulk-loading-data-into-mnesia//comment-page-1#comment-482 用Mochiweb打造百万级Comet应用，第三部分 - IDISC的生活 Thu, 22 Jan 2009 09:09:04 +0000 http://www.metabrew.com/?p=82#comment-482 [...] 注意这额外的mnesia参数 - 这是避免** WARNING ** Mnesia is overloaded 你可能在别的地方看到的警告信息。提到我以前发表的: On bulk loading data into Mnesia 有另外的调入大量数据的方法。最好的解决方案看起来是设置这些选项(在评论中指出的, 谢谢Jacob!) 。Mnesia 参考手册 在Configuration参数中包含了很多其他的设置，值得一看. [...] [...] 注意这额外的mnesia参数 – 这是避免** WARNING ** Mnesia is overloaded 你可能在别的地方看到的警告信息。提到我以前发表的: On bulk loading data into Mnesia 有另外的调入大量数据的方法。最好的解决方案看起来是设置这些选项(在评论中指出的, 谢谢Jacob!) 。Mnesia 参考手册 在Configuration参数中包含了很多其他的设置，值得一看. [...]

]]> By: Erlang-China » Comet and Erlang, A perfect match http://www.metabrew.com/article/on-bulk-loading-data-into-mnesia//comment-page-1#comment-124 Erlang-China » Comet and Erlang, A perfect match Sun, 30 Nov 2008 15:11:27 +0000 http://www.metabrew.com/?p=82#comment-124 [...] ** Mnesia is overloaded messages you would (probably) otherwise see. Refer to my previous post: On bulk loading data into Mnesia for alternative ways to load in lots of data. The best solution seems to be (as pointed out in the [...] [...] ** Mnesia is overloaded messages you would (probably) otherwise see. Refer to my previous post: On bulk loading data into Mnesia for alternative ways to load in lots of data. The best solution seems to be (as pointed out in the [...]

]]> By: A Million-user Comet Application with Mochiweb, Part 3 | Richard Jones, Esq. http://www.metabrew.com/article/on-bulk-loading-data-into-mnesia//comment-page-1#comment-53 A Million-user Comet Application with Mochiweb, Part 3 | Richard Jones, Esq. Wed, 05 Nov 2008 04:20:05 +0000 http://www.metabrew.com/?p=82#comment-53 [...] ** Mnesia is overloaded messages you would (probably) otherwise see. Refer to my previous post: On bulk loading data into Mnesia for alternative ways to load in lots of data. The best solution seems to be (as pointed out in the [...] [...] ** Mnesia is overloaded messages you would (probably) otherwise see. Refer to my previous post: On bulk loading data into Mnesia for alternative ways to load in lots of data. The best solution seems to be (as pointed out in the [...]

]]> By: Jacob http://www.metabrew.com/article/on-bulk-loading-data-into-mnesia//comment-page-1#comment-3 Jacob Tue, 14 Oct 2008 01:46:36 +0000 http://www.metabrew.com/?p=82#comment-3 You can tweak some settings to get rid of most of those overload messages. I used to get them all the time, before I set the following system level config options: {mnesia, [{dc_dump_limit, 40}, {dump_log_write_threshold, 50000}]} Increasing the dump_log_write_threshold means the transaction log is dumped less often, and increasing the dc_dump_limit means the disk table is dumped from the log more often. YMMV, Jacob You can tweak some settings to get rid of most of those overload messages. I used to get them all the time, before I set the following system level config options:

{mnesia, [{dc_dump_limit, 40}, {dump_log_write_threshold, 50000}]}

Increasing the dump_log_write_threshold means the transaction log is dumped less often, and increasing the dc_dump_limit means the disk table is dumped from the log more often.

YMMV,
Jacob

]]>