True DB Grid, SEF Data Source and DB Navigator

  • fpnzmarksh 2 years, 2 months ago

    We have a series of controls built with the True DB Grid, SEF Data Source and DB Navigator trinity.

    The DB Navigator and True DB Grid have their DataSource properties set to the SEF Data Source. The DataMember property is set to the appropriate name from the ViewSource collection.

    Underneath this is a .NET 4 EF EDMX that points to an Oracle Db via the ODP.NET 4 provider.

    The DB Navigator exposes FIRST/PREVIOUS/NEXT/LAST/REFRESH (these are read-only datasets).

    When a control is created, then visualised and the grid is populated with more than one row, we have discovered that the PREVIOUS button always navigates to row 2. Even when you are on row 2, clicking PREVIOUS appears to not change anything.

    The DB Navigator exposes a button-click event, but we are not handling the event.

    What we would like to establish is:

    1. Is this behaviour related specifically to the SEF data source?
    2. Is this behaviour related to the fact we are using a trial version to determine if the toolset will do what we need it to?
    3. What if anything should we be doing in order to have PREVIOUS always move to the previous row?

    When loading medium size sets of data (around 200,000 rows) we have also noticed that there is a large amount of memory consumed (over 1GB) and the load takes approx 1 minute per 50,000 rows.

    How can we improve performance on larger datasets?

  • 10p
    domsinclair10p 2 years, 2 months ago

    Hi

    This was a specific problem with one particular build of the db Navigator. If you run C1 live and update the c1 controls (specifically C1 Input) to the latest version and obviously then ensure that the latest version is referenced in your project, you should find that this particular issue goes away.

    Dom

  • 3p
    C1_NodirT3p 2 years, 2 months ago

    > the PREVIOUS button always navigates to row 2
    Yes, just as Dom said, there was a problem in DbNavigator. It is fixed now. Just use the latest version.

    > How can we improve performance on larger datasets?
    Do you load 50,000 or 200,000 at once?
    Try using VirtualMode or paging. The former dynamically fetches segments of data that must be displayed in a grid.

  • fpnzmarksh 2 years, 2 months ago

    {domsinclair} and {C1_NodirT}:
    Many thanks for the assistance. I’m really new to Component1 and your help has been truly invaluable.

    DB Navigator is working beautifully now!!! :-) )

    My apologies for not providing more detail about the performance aspect.

    The underlying RDBMS is a remote Oracle 11g and we use the Oracle ODP.NET (CLR4) as the .NET data provider. We understand the remote/network nature will slow overall performance.

    The data we are extracting is from a view that yields slightly over 210k rows. We do not perceive this as a "large" dataset.

    We have investigated VirtualMode (Managed) (thanks {C1_NodirT}!) through the C1 documentation and implemented a test against our "medium" size data set.

    The main hurdle we encountered is that it operates only with the "flexgrid", where we have exclusively made use of the "True DB Grid" throughout the application.

    We have made use of the "True DB" grid because of its features: load/save grid layout, filter bar/drop-down et al.

    We feel it is very unfortunate that VirtualMode is not available against the flagship grid from Component1.

    What is the timeframe for porting VirtualMode to the True DB Grid?

    Checking performance, the memory consumption is down to less than 300MB. The initial 4:30 time penalty is now "spread out" over the data you choose to review.

    We tried changing the default page size too 500 and there was a minor increase in memory consumption and time lags between pages.

    Where can the attributes of the cache be manipulated? We are looking especially for expiry/aging time or cache reluctance so we can trade off memory usage (higher) with performance (faster).

    Again, many, many thanks for all of your help.

  • fpnzmarksh 2 years, 2 months ago

    Another quick behaviour question.

    We are observing the same performance hit when scrolling right (more columns) as we see when scrolling down (more rows).

    This suggests the cache is only for the visible portion of the dataset.

    When VirtualMode is MANAGED and the extender property is set TRUE, is the cache strictly row based?

  • 10p
    domsinclair10p 2 years, 2 months ago

    Hi

    'We feel it is very unfortunate that VirtualMode is not available against the flagship grid from Component1.’

    I can think of one or two people who might beg to differ on the matter of flagship status of the respective grids! As you state yourself that you’re new to C1 perhaps a very brief bit of background might be useful here. C1 came about through the merger of two different companies, each of which had their own grid. Both were seen to have strenghths and as such both have continued to be developed. The obviously tight integration with data sources will often, on the face of it at least, make the true db grid seem like the obvious choice for use with any 'database’ relataed activity, and in all honesty that was very often the way that I would work in the past. Graduallt through trial and a lot of error on my part, I have come to appreciate the 'flexibility’ that the flex grid can offer, especially where using a grid in conjunction with the SEF is concerned. You ought to find that almost everything that you are currently doing with the True DB Grid you can do with the flex grid, and if you haven’t yet looked at the various samples for it then I would urge you do do so (if you happen to speak with braced c# tongue then you’ll be at a distinct advantage over me when it comes to those). It is also worth noting that the SEF is still a relatively ne product in the C1 stable and that to date more work has be done (in terms of getting features to work well) with the flex grid than with the true db grid.

    I’ll let Nodir address the more technical aspects of the Virtual mode, he really is leagues above me in terms of understanding the intracacies of that, but my understanding at least suggests that when you have virtual mode on the cache will comprise of the data that has thus far been retrieved (although there is a basic notion of how much underlying data there is), the more that is retrieved the bigger the cache and obviously once in the cache it remains there.

    I’m sorry that this doesn’t entirely answer all of your questions, but hopefully it’s a start.

    Dom

  • fpnzmarksh 2 years, 2 months ago

    Realise I’ve been very lax. I’m Mark.
    Apologies for the oversight.

  • 10p
    domsinclair10p 2 years, 2 months ago

    Mark

    No need to apologise, as I’d said earlier my natural inclination would always have been to vere towards the True DB grid (after all the clue’s in the name is it not?), but I have come to appreciate a lot of what can de done with the flex grid, and that is especially true when working with the SEF. For example sit back and look at the data that you are returning from the database and then look at how you can manipulate that with the flex grid. After a while you begin to see that the need to write a number of queries to hone the data is not always necessary. That’s not to say that it eliminates the need to further query data merely that it’s capable of doing some rather clever things with the pool (or cache) of data that you already have.

    One of the aspects of this that I have come to appreciate ids the ability to create a tabbed form containg a nunber of flexgrids, ranging from data entry to analysis of the same. The data cache that is provided by the SEF makes this sort of thing a breze, and then combine all of that with an olap panel and you’ll be rocking and rolling before you know it.

    Dom

  • 2p
    C1_MichaelE2p 2 years, 2 months ago

    @fpnzmarksh said:

    We feel it is very unfortunate that VirtualMode is not available against the flagship grid from Component1.

    What is the timeframe for porting VirtualMode to the True DB Grid?

    Actually, virtual mode does work with C1TrueDbGrid. There are two kinds of virtual mode, Managed and Unmanaged. Managed mode is not supported for C1TrueDbGrid, but Unmanaged mode is, and for C1TrueDbGrid Unmanaged mode is enough, it works fine with C1TrueDbGrid (to our knowledge). Managed mode is not necessarily better than Unmanaged, it’s just that Managed mode means that our code treats a particular control (grid) specifically. Some controls need such specific treatment and won’t work well without it (it depends on how the control is written). C1TrueDbGrid is not like that, it works fine with virtual mode without this special treatment (again, to our kowledge; if you find a problem with Unmanaged virtual mode with C1TrueDbGrid, please let us know).

  • 2p
    C1_MichaelE2p 2 years, 2 months ago

    @fpnzmarksh said:

    Where can the attributes of the cache be manipulated? We are looking especially for expiry/aging time or cache reluctance so we can trade off memory usage (higher) with performance (faster).

    There is only one setting for tuning, CacheTimeout, http://helpcentral.componentone.com/nethelp/c1studioEF/#!XMLDocuments/Reference/html/P_C1_Data_DataSource_ClientViewSource_CacheTimeout.htm
    It controls how often SEF checks for evicting unneeded entities from cache. But usually it does not make a big difference. It can be useful to change it only when SEF spends too much time deciding which entities can be safely evicted. Usually it does it fast enough so changing this setting is not necessary.

  • 2p
    C1_MichaelE2p 2 years, 2 months ago

    @fpnzmarksh said:
    Another quick behaviour question.

    We are observing the same performance hit when scrolling right (more columns) as we see when scrolling down (more rows).

    This suggests the cache is only for the visible portion of the dataset.

    When VirtualMode is MANAGED and the extender property is set TRUE, is the cache strictly row based?

    It’s strange that you observe that effect with scrolling right (more columns). Maybe it’s some grid behavior. The data source, SEF is not aware of horizontal scrolling, only vertical scrolling matters to it. The cache contains rows, always entire rows (more exactly, entities), not parts of entities.

  • 2p
    C1_MichaelE2p 2 years, 2 months ago

    @domsinclair said:
    …when you have virtual mode … the more that is retrieved the bigger the cache and obviously once in the cache it remains there.

    Dom, you probably did not mean that, it was probably a typo, but just in case: in virtual mode you can;t say that once in cache entities remain there. In virtual mode (and only in virtual mode) the cache does not keep all entities that were put there. It keeps only those that are needed, constantly checking which are needed (time interval for that check controlled by the CacheTimeout property). If it sees that an entity is not currently used by any bound control and is not modified, it expels it from the cache. That allows virtual mode to keep low (or at least reasonably bound) memory consumption even when the user browses through a huge data set fetching many thousands entities.

  • 10p
    domsinclair10p 2 years, 2 months ago

    >it was probably a typo

    Yes and no. As you probably know better than most I have a myriad of notes on various aspects of this both on my machine and various notebooks. In all honesty I was probably looking at the wrong bits whilst merrily typing away saying something else. I’m loathe to think of it as the onset of senior moments, but you can never be to sure!

    However thanks for clarying the issue.

    Dom

  • fpnzmarksh 2 years, 2 months ago

    Thank you for all of your support. As a newbie to Component One, I’m greatly appreciating the assistance. I’ll apologise now in case I ask anything that seems really silly.

    @c1_michaele said:
    … In virtual mode (and only in virtual mode) the cache does not keep all entities that were put there. It keeps only those that are needed, constantly checking which are needed…

    This was our understanding from the SEF documentation.

    @c1_michaele said:
    It’s strange that you observe that effect with scrolling right (more columns)…

    We have done some additional analysis relating specifically to the issue we observe with a "large-ish" set of entities.

    The result set is 210252 entities and each entity carries 79 properties (columns).

    The Microsoft .NET4 part of the Entity Framework encapsulated in the EDMX points to an Oracle 11g database, which we are using ODP.NET 4 to access.

    Our UI is a FlexGrid, DB Navigator and C1 Data Source (from SEF). The grid and navigator are bound to the data source and the same named entity (Data Member).

    The environment is Visual Studio 2010 on Windows XP Prof Version 2002 SP3. This runs on an Intel Pentium 4 at 3.2GHz with 1GB RAM.

    When we access the grid and page down using the vertical scrollbar, there is a 2-4 second delay. If we scroll horizontally, the same delay is observed.

    Specifically, we scrolled right twice, used the DB Navigator to travel to the LAST row in the grid (the column position remains unchanged) and then scrolled left once.

    Each navigation was accompanied by the 2-4 second delay.

    Navigating to FIRST was near instantaneous, because the required data is in cache.

    While testing, we also noticed that the FlexGrid filter drop-down is only populated with values that have been loaded into the cache. Not sure why this would be.

    We understood from the documentation that UNMANAGED was available in VirtualMode, but the documentation seems to suggest we need to do a great deal extra to achieve this.

    Is there a sample available that shows the True DB Grid interacting with a SEF Data Source in unmanaged virtual-mode?

    Mark

  • 2p
    C1_MichaelE2p 2 years, 2 months ago

    @fpnzmarksh said:
    The result set is 210252 entities and each entity carries 79 properties (columns).
    When we access the grid and page down using the vertical scrollbar, there is a 2-4 second delay. If we scroll horizontally, the same delay is observed.

    Maybe 79 properties is just a lot. Maybe it’s not related to virtual mode, maybe it just takes a lot of time (seconds) to fetch the data with all 79 properties (maybe some of the properties contain big strings, maybe some are even loaded deferred, "lazy loading" – the latter is hinted to by the fact that horizontal scrolling takes long time). You can try to find out how long does it take to fetch 2*N rows with all 79 properties, where N is the number of rows visible in your grid. You can do it without virtual mode, just make a filter that restricts the number of records returned by your query, and a form that is not immediately filled with data but is filled on your command (for example, make the filter initially return 9 records, then change the filter on a command, or do something else of that kind) and measure the time.

Viewing 15 posts - 1 through 15 (of 16 total)

You must be logged in to reply to this topic.