Question

When the query returns more than 5000 records, the plugin Saiku takes more than 7 minutes to return the data. This problem does not occur in JPivot. I'm using Pentaho BI-Server. Someone knows this problem?

Answer 1

1	I see tremendous improvement! Thanks Paul!! link answered 08 Feb, 05:14 Praveen DS 16●1●5 accept rate: 0% no problem :) glad you like it (09 Feb, 06:29) pstoellberger ♦♦

Answer 2

0

if you are returning 5000 records then you are probably doing something wrong why would you return so many records? are you trying to export them? or do you just want to display them in the UI?

link

answered 04 Jul '11, 09:15

pstoellberger ♦♦
897●2●7
accept rate: 5%

There is an attribute that generates a lot of records. The problem is that if you use this attribute incorrectly, the screen freezes. Example: The attribute 3 has a total of 56,000 members. It is usually used a hierarchy attribute 1, attribute 2, attribute 3, but if the user removes the attribute 2 and display the attribute 3 direct, the whole screen freezes.

Thanks

(04 Jul '11, 09:32) Leandro

does the screen freeze without the "executing query.... " message? could you copy me the last few tomcat/bin/saiku.log entries? right after you executed this query

(04 Jul '11, 09:42) pstoellberger ♦♦

Answer 3

oh sorry you are using the biserver plugin. could you add that to your tomcat/webapps/pentaho/WEB-INF/classes/log4j.xml

  <param name="File" value="saiku.log"/>
  <param name="Append" value="false"/>

  <!-- Rollover at midnight each day -->
  <param name="DatePattern" value="'.'yyyy-MM-dd"/>

  <!-- Rollover at the top of each hour
  <param name="DatePattern" value="'.'yyyy-MM-dd-HH"/>
  -->

  <layout class="org.apache.log4j.PatternLayout">
     <!-- The default pattern: Date Priority [Category] Message\n -->
     <param name="ConversionPattern" value="%d %-5p [%c] %m%n"/>

     <!-- The full pattern: Date MS Priority [Category] (Thread:NDC) Message\n
     <param name="ConversionPattern" value="%d %-5r %-5p [%c] (%t:%x) %m%n"/>
      -->
  </layout>

</appender>

Answer 4

0

It looks like formatting is taking enormous amount of time. Here is the debug trace with 2000+ records

2012-01-03 18:08:07,494 INFO [org.saiku.service.olap.OlapQueryService] Size: 2/2490 Execute: 837ms Format: 37135ms Total: 37972ms

I am using 2.2RC1 plugin.

link

answered 03 Jan, 05:15

chut
16●2●3●7
accept rate: 0%

edited 03 Jan, 05:16

Answer 5

The time degradation is not linear. Here is another dataset with 70 records:

2012-01-03 18:17:46,992 INFO [org.saiku.service.olap.OlapQueryService] Size: 2/70 Execute: 215ms Format: 50ms Total: 265ms

I have another dataset with some 12,000 records, and the execution won't finish ever after some 15 minutes.

Answer 6

0	as soon as the cache is filled it will be fast. we know that there are some issues with our formatters that slow down the whole process. this is definitely something we will work on as soon as possible! link answered 09 Jan, 14:18 pstoellberger ♦♦ 897●2●7 accept rate: 5%

Answer 7

0

i just found a huge issue in that code and i managed to speed it up big time i should have it commited in a few minutes so if you download the latest saiku from http://ci.analytical-labs.com you should see a performance boost for large queries

link

answered 18 Jan, 21:15

pstoellberger ♦♦
897●2●7
accept rate: 5%

Answer 8

0	The improvement is remarkable indeed. 16,000 rows is now rendered in the matter of seconds. Thanks for the great work. Chut link answered 19 Jan, 04:12 chut 16●2●3●7 accept rate: 0%

Slowness with large amounts of records

Follow this question

Related questions