Category Archives: Art of BI
Moving Large Datasets to Oracle EPM Cloud: How to Do It Right
From greater accessibility and flexibility to cheaper storage costs, migrating your financial data to Oracle EPM Cloud has many benefits—but the project needs careful handling. The challenges involved with moving large datasets to the cloud include:
- Selecting the right storage methodology; files, objects, and databases need different handling methods.
- Preparing and cleansing the data (e.g. removing scratch files and backups, or archiving and compressing large amounts of relatively small files).
- Optimizing network transfer speeds and replicating data to multiple regions or cloud providers.
- Scaling the data migration process and avoiding bottlenecks.
- Validating data integrity by checking for corruption and usability after the migration.
- Preparing the data for use once it’s in the cloud.
The good news is that moving large datasets to the cloud is eminently doable as long as you’re able to put in the time and effort—and if you work with the right cloud migration partner. Datavail recently helped one of our clients, a large operator of hundreds of gas stations and retail locations across North America, upgrade their aged Oracle Hyperion HFM deployment and migrate their enterprise performance management (EPM) data to the cloud.
The steps for managing migrations with datasets of this size can include (plan for adjustments based on organizational variety):
- Gathering requirements and performing in-depth analyses of the client’s IT systems.
- Identifying points of improvement while maintaining on-premises functionality. For example, Datavail decreased the number of custom dimensions from 4 down to 2, so that the client could use the standard-size Oracle Financial Consolidation and Close (FCCS) application.
- Building and converting the client’s metadata, including their alternate hierarchies. This included using the Oracle Smart View metadata feature, which is only available in the cloud, to build hierarchies much more quickly than on-premises.
- Discovering errors and inefficiencies within the client’s datasets and reporting functionality.
Datavail has helped more than 150 clients move their enterprise data to the cloud—whether it’s Oracle EPM Cloud like the client in this post, or Amazon Web Services, Microsoft Azure, OCI, or much more. To learn more about how our client saved hours of time during their financial consolidation processes, check out our case study Major Retailer Improves Productivity by Moving to Oracle EPM Cloud.”
Read This Next
4 Ways Datavail Prepares Companies for Oracle Hyperion EPM 11.1 End of Support
Staying on top of your enterprise software licenses and expiration dates is crucial—especially when it comes to mission-critical applications like Oracle Hyperion EPM. Download the paper to learn how to prepare for your EPM upgrade.
The post Moving Large Datasets to Oracle EPM Cloud: How to Do It Right appeared first on Datavail.
Stakeholder Management: The Key to Application Development Project Success
Stakeholder management is critical to the success of any application development project. Your team can produce an outstanding application, but if the people who have a vested interest in the application aren’t supportive, it’s just wasted effort.
Why You Need to Manage Stakeholders
A stakeholder is anyone who is affected by or can have an effect on an AppDev project. But, it’s important to note that the term stakeholder management is just an easy way to talk about working with stakeholders. In practice, stakeholder management has less to do with managing, and more to do with promoting involvement, communicating, identifying requirements, uncovering and addressing concerns, and coaching.
You need to pay attention to stakeholder management because you’ll receive a number of benefits:
- You’ll reduce risk:You may have been involved in or heard about an AppDev project that was scuttled at the last minute because of the actions of one of the stakeholders. Correctly identifying all the stakeholders and making sure that they and the project team are on the same page will reduce conflict and increase the chances of success.
- Your stakeholders will understand their role:You’ll be able to define the role for each type of stakeholder and they will be able to focus on how they can participate most effectively.
- Your stakeholders will be more involved:Stakeholders can find it difficult to carve time out of their schedules to participate in an AppDev project. With effective stakeholder management, your stakeholders will understand the benefits they’ll receive from participating. This will help them stay engaged and focused on ensuring the project’s success.
How to Design a Stakeholder Management Plan
The process used to manage stakeholders isn’t complicated, but getting it right is essential. There are four steps to follow.
- Identify
Start by identifying a project’s stakeholders. If you’re using the Agile methodology, start by talking to the Product Owner to explore all the people who may be affected by the project.
Internal stakeholders can reside in departments outside of the one you’re working with. Explore how the application you’re developing for one department may affect others. For example, find out if another department provides input that may need to change. Or, if the output from your application will go to another department, you’ll need a stakeholder from that department on your team to ensure a smooth flow of data.
External stakeholders are those people who don’t work for your company, but who still need to be considered. That group can include customers or people in your supply chain, for example.
- Assign Roles
Review the list of stakeholders you’ve identified and determine what role they will play in the project. Use the information you gathered during the Identification step, and wherever possible, interview each stakeholder to confirm their level of influence and interest.
- Prioritize
Not all stakeholders will have the same level of involvement with the finished application, or influence on your project. It’s important to prioritize your list of stakeholders to assign logical roles that give you the participation that you need while meeting the needs of the individuals. These categories will help with the prioritization.
- Group 1 – High influence and high interest: These are the key stakeholders such as Product Owners, project sponsors, or business leaders.
- Group 2 – High influence and low interest: These stakeholders may not be directly affected by the project, but they can influence its result.
- Group 3 – Low influence and high interest: This group can consist of lower-level employees who will be directly affected by the project, but don’t have direct influence. These individuals can make the difference between an enthusiastic work group or an unsupportive one.
- Group 4 – Low influence and low interest: These stakeholders may work in other departments not affected by the project, but they have an interest in how the project is progressing.
- Define Participation/Communication
The last step is to define how each group you’ve prioritized will participate in the project. Some stakeholders will only participate by reviewing project communications, while others will have highly active roles in the project itself.
For example, all stakeholders should receive regular communications from the project team. For those in Group 4, that may be all they’ll need to help them monitor progress. People in Group 2 may need a more active role in terms of providing input into planning sessions and attending sprint demos. People in Groups 1 and 3 will be extremely involved in the project and should participate as much as possible.
Best Practices for Engaging Stakeholders
It’s one thing to define roles for stakeholders and to get their agreement to participate in those roles. It’s another thing for them to follow through. Each stakeholder in an AppDev project has a busy schedule, and it’s easy for them to prioritize the project below their regular activities. These tips can help to provide the motivation stakeholders need to focus on an AppDev project.
- Reinforce the benefits
It’s human nature for people to pay attention to something that benefits them. Stakeholders in an AppDev project need to be reminded of the benefit the final product will offer, as well as the benefit to them of participating in the project.
You can use both formal and informal communication to reinforce those two types of benefits. For some stakeholders, the project may offer business growth opportunities and their participation may help them steer the project to meet business goals. For other stakeholders, the product may benefit them by making their work easier and their participation will ensure that the system addresses their needs.
Work to understand the stakeholders’ motivations and help them to see how the product and their participation offers them personal and professional benefits.
- Get them involved early and often
For those stakeholders where direct involvement is important, make sure they get involved in the early stages of the project. For example, if you’re using Agile methods, invite the right stakeholders to story mapping sessions to get them involved with the development team. You can also invite stakeholders to sprint demos so that they can see the results of their efforts and you can get their feedback.
- Establish trust
Stakeholders will need to go the extra mile to participate in the project. You’ll need to establish a culture of trust within the entire team to motivate stakeholders to put in the effort that’s required. You can build trust with stakeholders by being transparent and telling the truth, even when it’s not popular.
Don’t underestimate the trust issue. It can have the power to make your project a failure or increase the cost and difficulty of completing it.
Final Thoughts
Paying close attention to managing stakeholders will have a significant positive effect on the success of any project, including an AppDev project. If the project team gets too involved in the technical aspects of a project, they will lose sight of the fact that they’re building the application for the stakeholders. Without the support and participation of the stakeholders, the project becomes just a coding exercise.
This situation is only one of the things that can cause an application development project to fail. Our recent whitepaper entitled “4 Reasons Why Application Development Projects Fail” is designed to help you understand the most common reasons why projects fail, and it describes solutions that will help you ensure your project’s success.
The post Stakeholder Management: The Key to Application Development Project Success appeared first on Datavail.
Support Data Management with Datavail’s New MongoDB Backup Tool
Even small and medium businesses generate a LOT of data, much or all of which must be appropriately backed up to facilitate a successful recovery if needed. Unfortunately, the expansion of data types and the drive for digital transformation have made managing those backup and recovery functions especially difficult. There are just too many data types, formats, systems, etc., to accomplish a “simple” backup function, let alone one that allows a swift return to work using a recovered option.
For SMBs relying on MongoDB for their data management needs, one option to resolve this challenge creates another challenge: how to back up the MongoDB files fully without also having to purchase the MongoDB backup and recovery services embedded in MongoDB Ops Manager or MongoDB Cloud Manager.
Fortunately, Datavail has developed a new and supported data backup and recovery option for MongoDB that is fast, reliable, and very well priced. SMBs that need reliable, swift, comprehensive backups but don’t want to pay enterprise-sized fees will appreciate accessing the new opportunity.
Data Backup Is Critical to Corporate Health
Every organization now relies on its data and corporate information to push forward in its market lane. Systems information, supply chains, customers, etc., all contribute critical insights to corporate health and inform decisions for a brighter corporate future.
However, gathering and utilizing all that data also generates the requirement of managing it, storing it, and keeping it safe. Successful backup capacities are also necessities, as these assure leaders they’ll have a reasonably functioning database if something bad should happen to their main engine. Access to supported recovery services also assures them that a recovery is possible even when something bad does happen. Further, with new data breach threats emerging every day, ensuring that comprehensive backup and recovery systems are in place is as much an element of corporate risk management as it is data security.
Despite these realities, many IT departments are increasingly challenged to accomplish these backup and recovery tasks given the depth, breadth, and scope of today’s data pools. Traditional methods aren’t designed to provide all the security and functionality requirements demanded by today’s challenging data types, formats, and sources:
- The task of backing up the vast quantities of information that live in existing data pools is already daunting. Adding the constant flow of incoming information that arrives in unstructured forms and from newly engaged devices creates another level of chaos. Without dedicated thought and attention, insufficient backup systems may corrupt or lose critical corporate details, and those gaps won’t be discovered until after an issue compels a recovery attempt.
- In other cases, backed up and recovered files, etc., might not line up with existing application architecture requirements. Here, older versions of backups may not mesh well or at all with newer iterations of critical corporate programming.
- Failing to engage a high-quality backup capacity also opens the risk of cyber intrusions and crimes. Having a reliable, comprehensive second database available eliminates the possibility of suffering damages caused by cybercriminal threats like ransomware.
While the “big box” cloud providers offer impressive backup and recovery services, for most SMBs, the costs involved in accessing those are prohibitive. MongoDB, for example, offers excellent database backup and recovery options, but companies must purchase the Enterprise level of services to access it, and that may be beyond both their needs and their budget.
Introducing Datavail’s MongoDB Backup Tool
The data backup and recovery professionals at Datavail recognized the challenges posed to SMBs by both the heavy demand for reliable backups and the potentially burdensome cost of the same charged by bigger services providers. They developed the MongoDB Backup Tool to respond economically and comprehensively to those concerns and now offer a fully capable MongoDB backup option that doesn’t require access to either enterprise-grade MongoDB Operations Manager or MongoDB Cloud Manager.
Features of the Datavail Tool
- Because it’s designed to mimic the GPL Percona Backup for MongoDB (pbm), Datavail’s version provides the same functions and capabilities.
- Suited for MongoDB versions 3.0 through 4.4, the programming:
- Offers a hot backup.
- Is able to read from secondary servers.
- Allows for point-in-time recovery.
- Compresses the backup as it runs.
Datavail’s backup servicing process ensures that the backup function accomplishes precisely what’s expected: a high-quality, reliable, and supported version of every file, folder, data bit, and object:
- Connect to MongoDB through the pbm agent.
- Run “pbm config” command to configure the backup location.
- Trigger the pbm backup command to launch the process.
- Create pbm collections in the admin database.
- Gather relevant MongoDB data, including the version number, the replica set name, the primary nodes, etc.
- Run the backup:
- Create necessary backup metadata.
- Confirm the backup location.
- Shift the “start backup” function to “running backup.”
- Identify the “oplog” start position.
- Launch “mongodump.”
- Complete “mongodump.”
- Start “oplogdump.”
- Mark the backup as complete.
The Recovery and Restore service included in the MongoDB Backup Tool adds the establishment of the restoration process to the program. The software follows the backup process paths in reverse, including decompressing the backed-up files and oplogs to recover data when needed.
We’ve launched this new tool as a service in part of a recent project we completed for an online gaming company that saved costs and improved processes. If you’ve got challenges with MongoDB backups, come speak with our experts about this solution.
Read This Next
It’s Game Over for Lacking Data Access
One of Datavail’s customers, an online gaming company, had thousands of users wanting access to older games and older versions of games and was looking for reliable and easy access to game data. Read this case study to learn more about our custom MongoDB Backup Solution including the process implementation for the customer and the results gained.
The post Support Data Management with Datavail’s New MongoDB Backup Tool appeared first on Datavail.
The Five Elements of Your On-Premises MongoDB to AWS Migration Strategy
A MongoDB application migration requires coordination between multiple teams, stakeholders, and systems. You need a comprehensive plan to reduce the risk of project failure, allocate resources to support the migration, and remain on-schedule and on-budget.
When migrating from on-premises MongoDB to Amazon Web Services (AWS), you need to consider five key elements when putting together a successful strategy.
AWS Instance Type
The first decision to make is which AWS instance type works best for your MongoDB application. While Amazon EC2 has a variety of options, we recommend M4 or I3 for MongoDB. You also need to determine the amount of memory and storage for the instance.
M4
M4 instance types are well-balanced between compute, memory, and network resources. This general-purpose instance is a good starting point as it can support a wide range of use cases. You can choose between a 2.3 GHz Intel Xeon E5-2686 v4 processor or a 2.4 GHz Intel Xeon E5-2676 v3 processor on M4.
I3
I3 instance types are bare metal and optimized for high-transaction, low-latency databases, helping you achieve better performance with this type of MongoDB application. You get 15.2 TB of NVMe SSD instance storage, which delivers up to 25 Gbps of bandwidth and high IOPS while remaining affordable for many businesses.
You can directly access physical resources on these bare metal instances to gain greater control and optimization opportunities. These servers are powered by Intel Xeon E5-2686 v4 processors with 36 hyper-threaded cores.
High Availability Strategy
MongoDB uses a replica set to achieve high availability, protecting the database from natural disasters, power and network outages, and hardware failures. When you migrate your application to AWS, you need to select regions and availability zones that are best suited to maintaining high availability. AWS regions are self-explanatory — they are geographic regions for the data centers. Availability zones are specific locations within that region.
MongoDB developers recommend configuring your infrastructure to distribute your replica set across an odd number of data centers. This approach helps you eliminate a single point of failure by spreading replica members throughout many locations.
We recommend at least a five-member replica set for your production environments, with the following configuration:
- One primary: Your primary member in a MongoDB replica set is the only one that receives write operations.
- Four secondary: Your secondary members maintain copies of the primary member’s data set and are read-only.
The total for the replica set always needs to be an odd number. Arbiters are included in old version, but are now being deprecated by MongoDB, however, the total still needs to be odd.
Security Design on AWS
You can expand on MongoDB’s built-in security features through AWS solutions, making your application a challenging target for any malicious actors. The security features to consider in your AWS design include encryption, SSL, networking, firewalls, virtual private networks, and security groups.
MongoDB Migration Plan
Your MongoDB migration plan should cover each step of the process. What teams will assist in the migration, when do systems need to cut over to the cloud-based deployment, and how much will it disrupt your daily operations?
You have a dozen of these details to consider when you create a migration plan that best suits your company’s requirements. What happens if essential technical personnel go on vacation or become ill during the migration process? How do you effectively get buy-in from stakeholders and ensure that you have the budget to complete the migration? What do you hope to gain from moving this application to the cloud?
The more questions you address before your MongoDB migration, the better prepared you will be when the process is underway.
Backup and Recovery Strategy
Your on-premises backup and recovery strategy may not translate to a cloud-based environment, so it’s time to revisit your existing plan. Make sure that your preferred solutions work with AWS and then work properly following the migration.
Read This Next
Everything You Need to Know to Move Your On-Premises MongoDB Data Center to AWS
This white paper covers the specifics of moving your MongoDB database applications and analytics to the cloud, addressing the prerequisites, technical requirements and walking you through the process step-by-step.
The post The Five Elements of Your On-Premises MongoDB to AWS Migration Strategy appeared first on Datavail.
How Can You Control Scope Creep?
Application development projects are vulnerable to many things that can cause failure, such as underestimating resources, inaccurate time estimates, and more. One thing that can easily cause an AppDev project to fail is scope creep. You can control scope creep if you know how to recognize it, understand what causes it, and have plans for managing or avoiding it.
How to Recognize Scope Creep
Virtually every AppDev project starts out with a definition of the project goal, timing, budget, and deliverables. You can recognize scope creep when a new requirement is added to a project that wasn’t covered in the original project description.
Something that will have a big impact on scope is relatively easy to recognize, although you still need to manage it. It’s the combination of a number of seemingly harmless changes that are often the reason why projects get out of scope.
Who Causes Scope Creep?
Scope creep can be initiated in any number of ways and comes from both inside and outside of the AppDev project team.
-
People on the Team
If your team members aren’t all on the same page, you’re leaving yourself open to scope creep. A developer who isn’t crystal clear on what the requirements are can inadvertently go in a direction that doesn’t lead to the desired deliverables as they’ve been defined. Sometimes even a small deviation can cause trouble with timing and budget.
As developers work on a project, they start to feel invested in the outcome. If one of your developers decides that the project needs a slightly different approach, or just a small alteration in how the user interface works, you’ve got a problem.
Coming up with a better mousetrap isn’t a bad thing, but it’s critical that your team members can identify an idea that brings the project out of scope. The team members need to discuss the idea with the team leader to identify the effect it will have before going off on a tangent. One person going out of scope can get an entire team second-guessing what is in scope and what is not.
-
Project Stakeholders
If you do a good job of stakeholder management, the odds of stakeholders increasing the project’s scope is much lower. But, even when stakeholders seem to be on board, a senior stakeholder with an unspoken agenda can cause havoc.
-
Customers/Clients/Product Owners
It doesn’t matter what you call them, there’s always someone you’re responsible to in terms of the overall project. Scope creep caused by this person is usually easy to spot because you’re looking for the issue to arise.
This is the person you got approval from, the one who reviewed all of the project descriptions and should understand the scope of the project. But, it can still happen that this person will ask for revisions or changes without realizing the effect on timing and budget.
-
Users
Typically, you need to test the software with your users. User feedback can be important to make sure the final product meets your customers’ requirements. But, that feedback can also be fraught with out-of-scope suggestions. You’ll need to be careful about identifying revisions that you can do and stay within scope and suggestions that are truly going outside the scope of the project.
-
Others
In many situations, there are others who can affect your project. You must take input coming from or output going to another entity into account during the project definition. Any type of unrecognized dependency can cause scope creep very easily.
How to Control Scope Creep
Controlling scope creep works differently depending on the application development methodology you use.
Controlling Scope Creep Using the Waterfall Methodology
The waterfall methodology isn’t structured to be change friendly. The project is defined at the start and specific plans are laid for the life of the project. This methodology is linear and sequential, where each phase is completed and reviewed before the next phase begins. As a result, ad hoc changes are discouraged. There is just too much work to change the entire project plan in mid-stream.
If a product owner wants to make a major change, the project team can respond with alternatives such as adding the change in a future release. If it’s not possible to reach a negotiated agreement, then it’s critical that you define the impact on timing and budget and obtain agreement from the product owner to expand both.
Controlling Scope Creep Using the Agile Methodology
The agile methodology is structured to avoid scope creep wherever possible. Since agile approaches AppDev in small increments and focuses on collaboration in an iterative process, any “changes” that appear before the team has addressed how to build a particular piece of the project aren’t considered changes at all. But, that’s assuming that the change won’t create more work than was initially planned.
Agile teams typically add a 20% contingency in project plans because changes do happen as business needs change. Agile plans account for that fact by giving themselves the flexibility to adjust to changes rather than discourage them.
Some people think a project is out of scope if the final product differs from the original project definition. Using agile, that’s not an issue. Agile focuses on satisfying the customer. If the final product does that, even if it differs from the original description, the agile team has completed a successful project.
Naturally, there are times when a change can cause scope creep in an agile project. For example, if a software feature has been designed, tested, and delivered, and then someone wants to eliminate it in favor of a different feature, that’s definitely scope creep. Another example would be a change that would swap a big change for something small. You can’t add 100 hours of work and eliminate 10 hours of work without scope creep.
If a change will cause an increase in the budget or timeline, then you’re back to negotiating with the product owner to approve more resources. Scope creep is typically easier to control in an agile project. It all goes back to the communication that agile stresses with everyone associated with a project.
From the start of an agile project, anyone who might request a change should understand the difference between when the response will be, “No problem,” and when the term scope creep might waft into the discussion.
How to Avoid Scope Creep
Scope creep, as with most problems, is easier to avoid than fix. Here are some things that will help you avoid scope creep:
- Make sure you have a clear scope when the project starts
- Communicate that scope to everyone who may be affected by the project
- Establish close collaboration among the team members and everyone who may be affected by the project
- Prioritize features when the project starts—you’ll use that list when you’re trying to decide if a change should have precedence over an existing requirement
- Establish agreement on how the team will handle changes
- Strive for transparency—a new request doesn’t mean you’ve done something wrong, so be open to working with the requestor to help them understand the tradeoffs
Scope creep is just one of the issues that can cause an AppDev project to fail, but there are solutions to software development challenges. Read our recent whitepaper, “4 Reasons Why Application Development Projects Fail.” Not only will you learn more about the top four reasons why projects fail, you’ll also learn about seven solutions that can help you avoid the high cost of failed projects.
The post How Can You Control Scope Creep? appeared first on Datavail.
