In Quick notes during a fiddle-session I played around with a table that contained all possible values for a date column.
To fill it I used a FOR LOOP with an INSERT and a commit every 10000 records.
That took 9:37 minutes for 3.652.061 records.
That's terrible performance and nothing that needs to be accepted with SAP HANA!
I found a way to do the same much quicker:
Statement 'insert into rmanydates (select * from all_days)'
successfully executed in 6.120 seconds (server processing time: 4.630 seconds) - Rows Affected: 3652060 ;
Question 1:
The dates inserted above have not been pre-computed in this example.
How do I do this?
I also looked at the memory consumption for the stored date tuples.
In the blog post I just used a row store table, because I thought: well, there's not going to be any column store compression anyhow.
(you know, because column store compression mainly builds on compressing duplicate column values. But with every possible date once, there are no duplicates, so no compression - or is there?)
However, I managed to get the data loaded into a column store table and use a lot less memory.
Question 2:
How to get from
BEFORE
--- General ---
Total Memory Consumption (KB): 25.719
Number of Entries: 3.652.061
Size on Disk (KB): 23.152
Memory Consumption in Main Storage (KB): 25.700
Memory Consumption in Delta Storage (KB): 19
Estimated Maximum Memory Consumption (KB): 25.719
to
AFTER
--- General ---
Total Memory Consumption (KB): 1.645
Number of Entries: 3.652.061
Size on Disk (KB): 12.912
Memory Consumption in Main Storage (KB): 1.626
Memory Consumption in Delta Storage (KB): 19
Estimated Maximum Memory Consumption (KB): 1.645
Both tables were fully loaded when the space consumption was analyzed.
if you know how that works, put your answers into the comments section!
- Lars