CN117271482B - Historical data migration method and device, electronic equipment and storage medium - Google Patents
Historical data migration method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN117271482B CN117271482B CN202311571005.5A CN202311571005A CN117271482B CN 117271482 B CN117271482 B CN 117271482B CN 202311571005 A CN202311571005 A CN 202311571005A CN 117271482 B CN117271482 B CN 117271482B
- Authority
- CN
- China
- Prior art keywords
- page
- migrated
- historical data
- data
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000005012 migration Effects 0.000 title claims abstract description 203
- 238000013508 migration Methods 0.000 title claims abstract description 203
- 238000000034 method Methods 0.000 title claims abstract description 71
- 230000006978 adaptation Effects 0.000 claims abstract description 79
- 230000004048 modification Effects 0.000 claims description 12
- 238000012986 modification Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 11
- 238000005516 engineering process Methods 0.000 claims description 11
- 230000004044 response Effects 0.000 claims description 4
- 238000012545 processing Methods 0.000 abstract description 10
- 230000008676 import Effects 0.000 description 15
- 230000006870 function Effects 0.000 description 11
- 230000008569 process Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 101150027734 cript gene Proteins 0.000 description 4
- 230000003068 static effect Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a historical data migration method, a device, electronic equipment and a storage medium, which relate to the technical field of data processing and comprise the following steps: based on the access address information of the system to be migrated, entering a migration module of the system to be migrated; performing compatibility adaptation of a high-version operating system, a browser and a target system on page elements for data display in a migration module; according to the adapting result, determining historical data to be migrated from the system to be migrated, and importing the historical data to be migrated into a historical data migration tool; and responding to the data export instruction, exporting the historical data to be migrated to the target system so as to enable the user to keep the data interface of the system to be migrated on the data interface checked by the target system. According to the invention, the historical data migration among different architecture systems can be automatically and intelligently realized through the developed historical data migration tool, a new system developer is not required to be proficient in the source system data structure, and the new system developer is not required to analyze the source system data structure in a full quantity.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a method and apparatus for migrating historical data, an electronic device, and a storage medium.
Background
The historical data migration technology is used for carrying out data migration between different application platforms and different architecture databases, and provides assistance for the user to reserve historical information assets. Therefore, it is important to quickly and accurately migrate historical data in one system to another.
In the related art, first, data to be migrated is collected from a source database using a source connection handle pointing to the source database, and the data to be migrated is processed into a target database statement set executable by a target database, and then the target database statement set is transmitted to the target database using a target connection handle pointing to the target database, so that the target database executes the target database statement set to insert information included in the data to be migrated into the target database. Therefore, the purpose of migrating the historical data in the source database to the target database is achieved.
However, since the existing historical data migration method needs migration personnel to be very familiar with the system data structure of the source database, and the source database does not need to generate different source connection handles according to different configuration information at the same time to collect data to be migrated, the efficiency of historical data migration is low, and meanwhile, the intelligence is not high.
Disclosure of Invention
The invention provides a historical data migration method, a device, electronic equipment and a storage medium, which are used for solving the defects of low efficiency and low intelligence of historical data migration in the prior art, and the data migration work can be automatically and intelligently carried out through a developed historical data migration tool to realize the purpose of historical data migration among different architecture systems, so that a new system developer is not required to be proficient in the data structure of a source system, and the data structure of the source system is not required to be subjected to full analysis by the new system developer, thereby greatly improving the efficiency of historical data migration, and simultaneously greatly improving the automation and the intelligence of the historical data migration.
The invention provides a historical data migration method which is applied to a historical data migration tool developed by adopting a Python crawler technology, and comprises the following steps:
based on access address information of a system to be migrated, entering a migration module of the system to be migrated;
performing compatibility adaptation of a high-version operating system, a browser and a target system on page elements for data display in the migration module;
according to the adapting result, determining historical data to be migrated from the system to be migrated, and importing the historical data to be migrated into the historical data migration tool;
And responding to a data export instruction, exporting the historical data to be migrated to the target system so as to enable a user to keep the data interface of the system to be migrated on the data interface checked by the target system.
According to the historical data migration method provided by the invention, the compatibility adaptation of the high-version operating system, the browser and the target system is carried out on the page elements for data display in the migration module, and the method comprises the following steps:
determining a main page corresponding to the data record in the migration module, judging whether the main page has a sub page, and performing compatibility adaptation on sub page elements of the sub page first and then performing compatibility adaptation on main page elements of the main page when the main page has the sub page;
and carrying out compatibility adaptation on the main page element under the condition that the main page does not have the sub page.
According to the method for migrating the historical data provided by the invention, the method for importing the historical data to be migrated into the historical data migration tool comprises the following steps:
based on a preset importing rule, importing information of the historical data to be migrated; the preset importing rule comprises the steps of storing structured data in the historical data to be migrated into a relational database, and storing unstructured data in the historical data to be migrated into a server disk in an encrypted mode.
According to the method for migrating historical data provided by the invention, the judging whether the main page has sub pages comprises the following steps:
based on a pre-stored sub-page identifier and/or sub-page keywords, whether the main page has the sub-page is judged.
According to the historical data migration method provided by the invention, the compatibility adaptation is performed on the main page element or the sub page element, and the method comprises the following steps:
based on a preset compatibility adaptation rule, carrying out compatibility adaptation on the target page element; the target page element is the main page element or the sub page element;
the preset compatibility adaptation rule comprises that aiming at the target page element, a page style is modified into a compatible high-version browser, a page frame style is modified into a preset page frame compatible style, element centering codes are added aiming at preset element content, access paths of css files and js files in a target page are respectively modified into preset access modes, a storage mode of gif files is modified into a preset storage mode, a page modification button of the target page is shielded, an accessory opening mode is modified into a preset opening mode, and calling interface programs are reserved before and after loading the target page; the target page is a page corresponding to the target page element.
According to the method for migrating historical data provided by the invention, the access address information based on the system to be migrated enters a migration module of the system to be migrated, and the method comprises the following steps:
responding to user operation, and starting a browser corresponding to the browser configuration information based on the browser configuration information of the system to be migrated;
and automatically inputting the access address information aiming at the address input box of the browser, and entering the migration module based on the login information of the system to be migrated.
According to the historical data migration method provided by the invention, the method further comprises the following steps:
automatically exiting the system to be migrated based on a successful lead-in result, and closing the browser; and the successful lead-in result characterizes that the historical data to be migrated is completely led in.
The invention also provides a historical data migration device which is applied to a historical data migration tool developed by adopting a Python crawler technology, and the device comprises:
the migration module entering unit is used for entering the migration module of the system to be migrated based on the access address information of the system to be migrated;
the page element adapting unit is used for carrying out compatibility adaptation of a high-version operating system, a browser and a target system on page elements for data display in the migration module;
The historical data importing unit is used for determining historical data to be migrated from the system to be migrated according to the adapting result, and importing the historical data to be migrated into the historical data migration tool; the history data to be migrated comprises page information and attachment information in the adaptation result;
and the historical data export unit is used for exporting the historical data to be migrated to the target system in response to a data export instruction so as to enable a user to keep the data interface of the system to be migrated on the data interface checked by the target system.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the historical data migration method as described above when executing the program.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method of historic data migration as described in any of the above.
According to the historical data migration method, the device, the electronic equipment and the storage medium, a historical data migration tool developed by adopting a Python crawler technology firstly enters a migration module of a system to be migrated based on access address information of the system to be migrated, compatibility adaptation of a high-version operating system, a browser and a target system is carried out on page elements in the migration module, then historical data to be migrated is determined from the system to be migrated according to an adaptation result, and the historical data to be migrated is imported into the historical data migration tool, so that the historical data migration tool can export the stored data to be migrated into the target system when responding to a data export instruction. When the historical data to be migrated is exported to the target system, the user can still keep the data interface of the system to be migrated on the data interface checked by the target system, so that the developed historical data migration tool can automatically and intelligently develop data migration work, the purpose of historical data migration among different architecture systems is achieved, a new system developer is not required to be proficient in the data structure of the source system, and the new system developer is not required to perform full analysis on the data structure of the source system, so that the efficiency of historical data migration is greatly improved, and meanwhile, the automation and the intelligence of historical data migration are also greatly improved.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a method for migrating historical data according to the present invention;
FIG. 2 is a second flow chart of the method for migrating historical data according to the present invention;
FIG. 3 is a schematic diagram of a historical data migration tool according to the present invention;
fig. 4 is a schematic structural diagram of an electronic device provided by the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The method, apparatus, electronic device and storage medium for migrating history data according to the present invention are described below with reference to fig. 1 to fig. 4, wherein an execution body of the method for migrating history data is a history data migration tool developed by using Python crawler technology, the history data migration tool may be installed on a terminal device or a server, and the history data migration tool may be displayed on the terminal device or the server in an application manner, and the history data migration tool has at least the same login function, browser opening and closing function, compatibility adaptation function, information storage function, information encryption function and data processing function.
Further, the terminal device may be a personal computer (Personal Computer, PC), a portable device, a notebook computer, a smart phone, a tablet computer, a portable wearable device, or other electronic devices; the server may use a stand-alone server or a server cluster composed of multiple servers, for example, the server may be a physical server including a stand-alone host, a virtual server carried by a host cluster, a cloud server, or the like. The present invention is not limited to the specific form of the terminal device or the server. The following method embodiments are described by taking a historical data migration tool developed by using a Python crawler technology as an execution subject.
Referring to fig. 1, one of the flow diagrams of the historical data migration method provided by the embodiment of the invention is shown in fig. 1, and the historical data migration method includes the following steps:
step 110, entering a migration module of the system to be migrated based on the access address information of the system to be migrated.
The system to be migrated can be a source storage database or a source barrel. This time is not particularly limited. Moreover, the migration module may be a module containing historical data to be migrated, for example, the migration module may be an inbox, an outbox, and the like.
Specifically, the historical data migration tool firstly opens a browser of the historical data migration tool through a preset browser driver, and then executes login operation of logging in the system to be migrated after access address information of the system to be migrated is automatically input, so that the system to be migrated can conveniently enter a migration module of the system to be migrated.
And 120, performing compatibility adaptation of a high-version operating system, a browser and a target system on page elements for data presentation in the migration module.
The page element may be a page element of a main page or a page element of a sub page, and the page element may include, but is not limited to, text, picture, color, typesetting, multimedia, and the like.
Specifically, in the case that the migration module is a module containing historical data to be migrated, the migration module may include at least one page and each page contains at least one data record, each data record may correspondingly enter a main page when receiving a click operation of a user, and at this time, the historical data migration tool may perform compatibility adaptation of a high-version operating system, a browser and a target system on page elements of the main page; performing compatibility adaptation of all data records of the current page in this manner; when the migration module comprises at least two pages, if all the data records of the current page are subjected to compatibility adaptation, the migration module can automatically turn pages sequentially under the action of a historical data migration tool and continuously perform the compatibility adaptation of all the data records of the next page; and performing compatibility adaptation until all data records of each page in the migration module are completely matched.
And 130, determining historical data to be migrated from the system to be migrated according to the adapting result, and importing the historical data to be migrated into a historical data migration tool.
The to-be-migrated historical data may include, but is not limited to, common field information, page information and attachment information, and the adaptation result characterizes that all page elements in the migration module have completed compatibility adaptation.
Specifically, after compatibility adaptation is performed on all data records of each page in the migration module, the historical data migration tool may determine to-be-migrated historical data from the adaptation result, that is, determine common field information, page information, attachment information and the like in the to-be-migrated system as to-be-migrated historical data, and further import the determined to-be-migrated historical data into the historical data migration tool.
In the process of importing the historical data to be migrated into the historical data migration tool, a developer corresponding to the target system can manually judge whether the historical data migrated into the historical data migration tool is abnormal in real time or periodically, and when judging that the migrated historical data A is abnormal, the historical data migration tool can be considered to be perfect in function, at the moment, the historical data migration tool after the function is perfect can migrate from the historical data A, and the historical data before the historical data A does not need to be migrated again.
And 140, responding to the data export instruction, exporting the historical data to be migrated to the target system so that the data interface of the system to be migrated is reserved on the data interface checked by the user on the target system.
The target system is pre-established with a historical data query module which is used for storing different types of historical data in different catalogues.
Specifically, because the historical data to be migrated is stored in the historical data migration tool, when the historical data migration tool receives the data export instruction, the historical data to be migrated can be exported to the target system, and the historical data to be migrated exported to the target system is subjected to adaptation processing and association processing. The adaptation processing is specifically to carry out compatibility adaptation on static resources such as style files, picture files and the like of the system to be migrated, so that compatibility with high-version browsers and different operating systems is ensured; the association processing here may specifically be that, for unstructured data and structured data respectively displayed in a list in the data to be migrated, association processing is performed according to the information of the identification code (Identity document, ID). The adaptation process and the association process may be performed automatically by the target system or manually by a developer who develops the target system. The present invention is not particularly limited herein.
Furthermore, the data to be migrated after the adaptation processing and the association processing can be imported into a historical data query module of the target system, and the historical data to be migrated is stored into different catalogues under the historical data query module in the target system. For example, static resources such as style files, picture files and the like of a system to be migrated in the historical data to be migrated are placed in a static data catalog of the historical data query module. The import operation imported into the target system may be automatically performed by the target system or may be manually performed by a developer who develops the target system. The present invention is not particularly limited herein.
Furthermore, the target system can assign rights to the imported data to be migrated according to a preset rights control mode, so that service users with different rights can be ensured to view the historical data of the corresponding rights.
According to the historical data migration method provided by the invention, the historical data migration tool developed by adopting the Python crawler technology firstly enters the migration module of the system to be migrated based on the access address information of the system to be migrated, and performs compatibility adaptation of the high-version operating system, the browser and the target system on page elements in the migration module, then determines the historical data to be migrated from the system to be migrated according to the adaptation result, and imports the historical data to be migrated into the historical data migration tool, so that the historical data migration tool can export the stored data to be migrated into the target system when responding to the data export instruction. When the historical data to be migrated is exported to the target system, the user can still keep the data interface of the system to be migrated on the data interface checked by the target system, so that the developed historical data migration tool can automatically and intelligently develop data migration work, the purpose of historical data migration among different architecture systems is achieved, a new system developer is not required to be proficient in the data structure of the source system, and the new system developer is not required to perform full analysis on the data structure of the source system, so that the efficiency of historical data migration is greatly improved, and meanwhile, the automation and the intelligence of historical data migration are also greatly improved.
It will be appreciated that the specific implementation of step 110 may include:
firstly, responding to user operation, and starting a browser corresponding to browser configuration information based on the browser configuration information of a system to be migrated; then, aiming at an address input box of the browser, access address information is automatically input, and the access address information enters a migration module based on login information of a system to be migrated.
Specifically, when the history migration tool displays the history migration tool and the terminal equipment in an application program mode, when the history migration tool receives user operation, the user operation can be responded, a browser corresponding to the browser configuration information is started based on the browser configuration information of the system to be migrated, such as a google browser, a firefox browser or a network seeker (Internet Explorer, IE) browser, when the browser is started, the history migration tool can automatically locate an address input box, automatically input access address information of the system to be migrated into the address input box, and then input login information of the system to be migrated into a popped login prompt box, so that the purpose of logging in the system to be migrated is achieved, and after logging in is successful, the system to be migrated enters a migration module of the system to be migrated.
It should be noted that, for the login information of the system to be migrated in the pop-up login prompt box, the input operation may be automatically performed by the history migration tool, or may be manually input by the user using the software. The present invention is not particularly limited herein. In addition, the login information of the system to be migrated may be account information and password information for logging in the system to be migrated, so as to verify whether the user logged in the system to be migrated is a user with login authority.
According to the historical data migration method provided by the invention, the historical data migration tool realizes the high efficiency and reliability of entering the migration module by entering the migration module of the system to be migrated based on the browser configuration information and the access address information of the system to be migrated.
It may be appreciated that, after step 120, the method for migrating historical data provided by the present invention may further include:
and automatically exiting the system to be migrated based on the successful result of the importing, and closing the browser.
The successful result of the importing characterizes that the historical data to be migrated is imported.
Specifically, when the history migration tool determines that the introduction operation of the history data to be migrated is completed, the successful introduction result representing the successful introduction can be determined, and at this time, the history migration tool can automatically exit the system to be migrated and automatically close the browser.
According to the historical data migration method provided by the invention, the historical data migration tool improves the intelligence and flexibility of the historical data migration tool in a manner of automatically exiting the system to be migrated and automatically closing the browser when the system to be migrated is determined to complete the importing operation of the data to be migrated.
It will be appreciated that the specific implementation of step 120 may include:
determining a main page corresponding to the data record in the migration module, judging whether the main page has sub pages, and under the condition that the main page has the sub pages, firstly carrying out compatibility adaptation on the sub page elements of the sub pages, and then carrying out compatibility adaptation on the main page elements of the main page;
in the case where there is no sub page in the main page, compatibility adaptation is performed on the main page element.
The main page elements can include, but are not limited to, characters, pictures, colors, typesetting, multimedia and the like in the main page, and the sub page elements can include, but are not limited to, characters, pictures, colors, typesetting, multimedia and the like in the sub page.
Specifically, the migration module may include at least one data record, where each data record may correspondingly enter a main page after responding to a click operation instruction of the user; when the migration module contains one data record, whether the main page corresponding to the data record has a sub page or not can be judged; when the migration module contains a plurality of data records, the plurality of data records can be displayed in one page or in a plurality of pages, the plurality of data records in each page can be displayed in a data line mode, and whether the main page corresponding to each data record in the data line mode contains sub pages or not can be judged sequentially.
Further, under the condition that the main page corresponding to the data record has the sub-page, compatibility adaptation is firstly carried out on sub-page elements of the sub-page, and then compatibility adaptation is carried out on the main page elements of the main page; when the main page does not exist, performing compatibility adaptation only on main page elements of the main page; compatibility adaptation for one data record is completed at this time. Compatibility adaptation of the next data record is then performed. And completing compatibility adaptation of all data records in the migration module.
According to the historical data migration method provided by the invention, the historical data migration tool realizes the purpose of carrying out compatibility adaptation on all page elements in the migration module by carrying out compatibility adaptation on the main page elements of the main page or carrying out compatibility adaptation on the sub page elements of the sub page and then carrying out compatibility adaptation on the main page elements of the main page aiming at the main page corresponding to the data record in the migration module, thereby improving the accuracy and reliability of the historical data migration and improving the efficiency of the historical data migration.
It may be appreciated that, in step 130, the history data to be migrated is imported into the history data migration tool, and the specific implementation process may include:
And carrying out information importing on the historical data to be migrated based on a preset importing rule.
The method comprises the steps of presetting an import rule, wherein the import rule comprises the steps of storing structured data in historical data to be migrated into a relational database, and encrypting unstructured data in the historical data to be migrated into a server disk.
Specifically, when information import is executed by the history migration tool, information import can be performed based on a preset import rule, namely, unstructured data such as sub-pages and attachments in history data to be migrated are stored in a server disk, structured data in the history data to be migrated are stored in a relational database, an attachment name in the unstructured data is encrypted by adopting a Message Digest algorithm (MD5), and sub-pages and attachment contents in the unstructured data are encrypted by adopting an SM4 or advanced encryption standard (Advanced Encryption Standard, AES) encryption algorithm; therefore, the purpose of importing the data to be migrated into the historical data migration tool is achieved.
According to the historical data migration method, the historical data migration tool improves the safety of historical data migration through a storage mode that structured data in data to be migrated is stored in the relational database and unstructured data is stored in a server disk in an encrypted mode.
It can be understood that, the specific implementation process of determining whether the main page corresponding to the data record has the sub page may include:
based on the sub-page identifier and/or sub-page keywords stored in advance, whether the sub-page exists in the main page corresponding to the data record is judged.
Specifically, the sub page identifier and/or the sub page keyword are pre-stored in the history migration tool, when the history migration tool enters the main page corresponding to the data record, the identifier and/or the keyword in the main page can be extracted, and the extracted identifier and/or keyword are respectively matched with the sub page identifier and/or the sub page keyword, so that whether the sub page exists in the main page corresponding to the data record is judged.
It should be noted that, for each identified sub-page, the historical data migration tool may store each sub-page and its corresponding sub-page key and/or sub-page identifier correspondingly. So that it can be exported subsequently in excel format or sql format.
According to the historical data migration method provided by the invention, the historical data migration tool judges whether the main page has the sub page or not based on the sub page identifier and/or the sub page keyword which are stored in advance, so that the functional diversity and the functional intelligence of the historical data migration tool are improved.
It may be appreciated that the compatibility adaptation of the main page element or sub page element may include:
based on a preset compatibility adaptation rule, carrying out compatibility adaptation on the target page element; the target page element may be a main page element or a sub page element;
the preset compatibility adaptation rule comprises that a page style is modified into a compatible high-version browser aiming at a target page element, a page frame style is modified into a preset page frame compatible style, element centering codes are added aiming at preset element content, access paths of css files and js files in the target page are respectively modified into preset access modes, a storage mode of gif files is modified into a preset storage mode, a page modification button of the target page is shielded, an accessory opening mode is modified into a preset opening mode, and calling interface programs are reserved before and after loading the target page; the target page is a page corresponding to the target page element, for example, the target page may be a main page or a sub page.
Specifically, the history migration tool modifies the page frame style to a preset page frame compatible style, for example, the page frame style is "border:1 sol", which can be normally displayed on the IE browser of the system to be migrated, but the target system cannot be compatible with the page frame style by using the google browser, and at this time, "border:1 sol" can be modified to "border:1px sol".
The history migration tool may add an element centering code for a preset element content, where the preset element content may be an element content in the target page element that can be centrally displayed on an IE browser of the system to be migrated and cannot be centrally displayed in a browser of the target system, and the element centering code is added for the preset element content. For example, the element centering code is editgtz3.Setattribute ("align", "center"). Therefore, the aim of modifying the page elements of the historical data in the system to be migrated is fulfilled, and the viewing requirements of a browser with a higher version and different operating systems are met.
The history migration tool can modify the access paths of the css file and the js file into preset access modes respectively. The cs file and js file are migrated to the target system, and page patterns of the cs file and js file cannot be displayed, so that the storage mode of the cs file and js file can be respectively modified to be a preset access mode of.
The history migration tool may modify the storage manner of the gif file to a preset storage manner. For example, the picture address in the target page element is stored in a manner of per oa/images/xxx.gif, and the target system cannot find the picture file corresponding to the picture address, so that the storage manner of the gif file/oa/images/xxx.gif can be modified to a preset storage manner of per. The id purpose of modifying the open mode of viewing the attachment of the historical data in the system to be migrated is achieved, and therefore the related attachment can be opened conveniently in a mode of viewing the attachment by using the target system.
The history migration tool may mask the page modification buttons of the target page, where the page modification buttons may be a modification button, a save button, and the like, so that the page modification buttons that exist when the target page is opened are not displayed in the target system.
The history migration tool can modify the accessory opening mode into a preset opening mode, and the modification process can be realized by referring to the following codes:
driver. Executejscript (' dimensions [0]. SetAttribute ] ("style", "display:"); dimensions [0]. SetAttribute ] ("onclick", "newoaopeneattachment ] (" tmpdoced+ ' dg wtps ' + '/'); ";) and;" ckzw [ tmpzw ]). The newaapplication method is implemented in a call interface program reserved before and after page loading.
The history migration tool can respectively reserve calling interface programs before and after loading the target page, and the reserved calling interface programs can be a startopendocument. Js interface program and an endopendocument. Js interface program and are used for writing JavaScript codes to realize functions in the target system. Illustratively, the reservation call interface may include:
res=res.replace('<head>','<head><script
type="text/javascript">document.write("<s"+"cript
type=\'text/javascript\'src=\''+pathtop+'/oanew/startOpenDocument.jsv="+Math.random()+"\'></s"+"cript>");</script>',1)；
res=res.replace('</html>','<script
type="text/javascript">document.write("<s"+"cript
type=\'text/javascript\'src=\''+pathtop+'/oanew/endOpenDocument.jsv="+Math.random()+"\'></s"+"cript>");</script></html>')。
it should be noted that, by modifying the history data to check the page and reserve the front and back loading interfaces of the page, it is ensured that the target system can develop new functions according to the service requirements, such as: the data checking log function is realized, and the data checking record migrated to the new system is ensured to be auditable; and the authority control of the same data and different user visible ranges is realized.
According to the historical data migration method provided by the invention, the historical data migration tool modifies the page style in the main page element or the sub page element into the compatible high-version browser, modifies the page frame style into the compatible style of the preset page frame, adds element centering codes for preset element contents, modifies the access paths of css files and js files in the target page into the preset access mode respectively, modifies the storage mode of gif files into the preset storage mode, shields the page modification type buttons of the target page, modifies the opening mode of the accessory into the preset opening mode and reserves the mode of calling interface programs before and after loading the target page respectively, so that the compatibility adaptation of the high-version operating system, the browser and the target system on the page element for data display in the migration module is completed, the target system can display original page data in the system to be migrated, and the display style of the system to be migrated can be reserved with the data difference of each stage, and the accuracy of the historical data migration is improved.
Referring to fig. 2, a second flow chart of a historical data migration method provided by the present invention, as shown in fig. 2, includes:
Step 210, responding to the user operation, starting a browser corresponding to the browser configuration information based on the browser configuration information of the system to be migrated, and then entering step 220.
Step 220, for the address input box of the browser, address information is automatically input, and based on the login information of the system to be migrated, the method enters a migration module, and then step 230 is entered.
Step 230, determining that the migration module contains at least one page of the data record, determining a main page corresponding to the data record, and then proceeding to step 240.
Step 240, judging whether a sub page exists in the main page, if so, entering step 250; otherwise, step 260 is entered.
Step 250, compatibility adaptation is performed on sub-page elements of the sub-page, and step 260 is entered.
Step 260, performing compatibility adaptation on the main page element of the main page, and proceeding to step 270.
Step 270, determining historical data to be migrated from the system to be migrated based on the adapting result, and entering step 280.
Step 280, performing information import on the historical data to be migrated based on a preset import rule, and entering step 290.
Step 290, judging whether the page where the data record is located is processed, if not, returning to step 230; otherwise, step 2100 is entered.
Step 2100, judging whether the page where the data record is located is the last page of the migration module, if so, entering step 2110; otherwise, after the page is automatically turned, the process returns to step 230.
Step 2110, automatically exiting the system to be migrated, and closing the browser.
The specific processes and technical effects involved in steps 210 to 2110 may be referred to the foregoing embodiments. And will not be described in detail herein.
The following describes a historical data migration device provided by the present invention, and the historical data migration device described below and the historical data migration method described above can be referred to correspondingly.
The historical data migration device provided by the invention is applied to a historical data migration tool developed by adopting a Python crawler technology. Referring to fig. 3, a schematic structural diagram of a historical data migration apparatus provided by the present invention, as shown in fig. 3, the historical data migration apparatus 300 includes: the migration module enters the unit 310, the page element adapting unit 320, the history data importing unit 330 and the history data exporting unit 340.
The migration module entering unit 310 is configured to enter a migration module of the system to be migrated based on access address information of the system to be migrated.
The page element adapting unit 320 is configured to adapt compatibility of the high-version operating system, the browser, and the target system to the page element for data presentation in the migration module.
The history data importing unit 330 is configured to determine the history data to be migrated from the system to be migrated according to the adapting result, and import the history data to be migrated into the history data migration tool.
The historical data export unit 340 is configured to export the historical data to be migrated to the target system in response to the data export instruction, so that the user retains the data interface of the system to be migrated at the data interface viewed by the target system.
It can be understood that the page element adapting unit 320 may be specifically configured to determine a main page corresponding to the data record in the migration module, determine whether the main page has a sub page, and in the case that the main page has the sub page, perform compatibility adaptation on the sub page element of the sub page first, and then perform compatibility adaptation on the main page element of the main page; in the case where there is no sub page in the main page, compatibility adaptation is performed on the main page element.
It may be appreciated that the page element adapting unit 320 may be further configured to determine whether the main page has a sub page based on a pre-stored sub page identifier and/or sub page keyword.
It can be appreciated that the page element adapting unit 320 may be further specifically configured to perform compatibility adaptation on the target page element based on a preset compatibility adaptation rule; the target page element is a main page element or a sub page element; the preset compatibility adaptation rule comprises that a page style is modified into a compatible high-version browser aiming at a target page element, a page frame style is modified into a preset page frame compatible style, element centering codes are added aiming at preset element content, access paths of css files and js files in the target page are respectively modified into preset access modes, a storage mode of gif files is modified into a preset storage mode, a page modification button of the target page is shielded, an accessory opening mode is modified into a preset opening mode, and calling interface programs are reserved before and after loading the target page; the target page is a page corresponding to the target page element.
It may be appreciated that the history data importing unit 330 may be specifically configured to import information into the history data to be migrated based on a preset importing rule; the method comprises the steps of presetting an import rule, wherein the import rule comprises the steps of storing structured data in historical data to be migrated into a relational database, and encrypting unstructured data in the historical data to be migrated into a server disk.
It may be appreciated that the migration module entering unit 310 may specifically be configured to, in response to a user operation, open a browser corresponding to the browser configuration information based on the browser configuration information of the system to be migrated; and automatically inputting access address information aiming at an address input box of the browser, and entering a migration module based on login information of a system to be migrated.
It may be appreciated that the migration module entering unit 310 may be further specifically configured to automatically exit the system to be migrated and close the browser based on the result of successful import; the successful result of the importing characterizes that the historical data to be migrated is imported.
The history data migration apparatus 300 provided by the present invention may execute the technical scheme of the history data migration method in any of the above embodiments, and the implementation principle and beneficial effects of the history data migration method are similar to those of the history data migration method, and reference may be made to the implementation principle and beneficial effects of the history data migration method, and no further description is given here.
Fig. 4 illustrates a physical schematic diagram of an electronic device, as shown in fig. 4, which may include: processor 410, communication interface 420, memory 430 and communication bus 440, wherein processor 410, communication interface 420 and memory 430 communicate with each other through communication bus 440. The processor 410 may invoke logic instructions in the memory 430 to perform a historical data migration method comprising:
Based on the access address information of the system to be migrated, entering a migration module of the system to be migrated; performing compatibility adaptation of a high-version operating system, a browser and a target system on page elements for data display in a migration module; according to the adapting result, determining historical data to be migrated from a system to be migrated, and importing the historical data to be migrated into a historical data migration tool; and responding to the data export instruction, exporting the historical data to be migrated to the target system so as to enable the user to keep the data interface of the system to be migrated on the data interface checked by the target system.
Further, the logic instructions in the memory 430 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a Read-only memory (ROM), a random access memory (RAM, randomAccessMemory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, the computer program product comprising a computer program, the computer program being storable on a non-transitory computer readable storage medium, the computer program, when executed by a processor, being capable of performing the historical data migration method provided by the methods described above, the method comprising:
based on the access address information of the system to be migrated, entering a migration module of the system to be migrated; performing compatibility adaptation of a high-version operating system, a browser and a target system on page elements for data display in a migration module; according to the adapting result, determining historical data to be migrated from a system to be migrated, and importing the historical data to be migrated into a historical data migration tool; and responding to the data export instruction, exporting the historical data to be migrated to the target system so as to enable the user to keep the data interface of the system to be migrated on the data interface checked by the target system.
In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform a method of migration of historical data provided by the methods described above, the method comprising:
Based on the access address information of the system to be migrated, entering a migration module of the system to be migrated; performing compatibility adaptation of a high-version operating system, a browser and a target system on page elements for data display in a migration module; according to the adapting result, determining historical data to be migrated from a system to be migrated, and importing the historical data to be migrated into a historical data migration tool; and responding to the data export instruction, exporting the historical data to be migrated to the target system so as to enable the user to keep the data interface of the system to be migrated on the data interface checked by the target system.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (8)
1. A method for migrating historical data, applied to a historical data migration tool developed using Python crawler technology, the method comprising:
based on access address information of a system to be migrated, entering a migration module of the system to be migrated;
performing compatibility adaptation of a high-version operating system, a browser and a target system on page elements for data display in the migration module;
according to the adapting result, determining historical data to be migrated from the system to be migrated, and importing the historical data to be migrated into the historical data migration tool;
responding to a data export instruction, exporting the historical data to be migrated to the target system so as to enable a data interface viewed by a user on the target system to keep the data interface of the system to be migrated;
the compatibility adaptation of the high-version operating system, the browser and the target system for the page elements for data display in the migration module comprises the following steps:
determining a main page corresponding to the data record in the migration module, judging whether the main page has a sub page, and performing compatibility adaptation on sub page elements of the sub page first and then performing compatibility adaptation on main page elements of the main page when the main page has the sub page;
Performing the compatibility adaptation on the main page element when the sub page does not exist in the main page;
the compatibility adaptation of the main page element or the sub page element includes:
based on a preset compatibility adaptation rule, carrying out compatibility adaptation on the target page element; the target page element is the main page element or the sub page element; the preset compatibility adaptation rule comprises that aiming at the target page element, a page style is modified into a compatible high-version browser, a page frame style is modified into a preset page frame compatible style, element centering codes are added aiming at preset element content, access paths of css files and js files in the target page are respectively modified into preset access modes, a storage mode of gif files is modified into a preset storage mode, a page modification button of the target page is shielded, an accessory opening mode is modified into a preset opening mode, and calling interface programs are reserved before and after loading the target page; the target page is a page corresponding to the target page element.
2. The method for migrating data according to claim 1, wherein importing the historical data to be migrated into the historical data migration tool comprises:
Based on a preset importing rule, importing information of the historical data to be migrated; the preset importing rule comprises the steps of storing structured data in the historical data to be migrated into a relational database, and storing unstructured data in the historical data to be migrated into a server disk in an encrypted mode.
3. The method of migrating historical data according to claim 1, wherein said determining whether there is a sub-page of the main page comprises:
based on a pre-stored sub-page identifier and/or sub-page keywords, whether the main page has the sub-page is judged.
4. A method of migrating historical data according to any one of claims 1 to 3, wherein entering the migration module of the system to be migrated based on access address information of the system to be migrated comprises:
responding to user operation, and starting a browser corresponding to the browser configuration information based on the browser configuration information of the system to be migrated;
and automatically inputting the access address information aiming at the address input box of the browser, and entering the migration module based on the login information of the system to be migrated.
5. The method of historical data migration of claim 4, further comprising:
automatically exiting the system to be migrated based on a successful lead-in result, and closing the browser; and the successful lead-in result characterizes that the historical data to be migrated is completely led in.
6. A historical data migration apparatus, for use with a historical data migration tool developed using Python crawler technology, the apparatus comprising:
the migration module entering unit is used for entering the migration module of the system to be migrated based on the access address information of the system to be migrated;
the page element adapting unit is used for carrying out compatibility adaptation of a high-version operating system, a browser and a target system on page elements for data display in the migration module;
the historical data importing unit is used for determining historical data to be migrated from the system to be migrated according to the adapting result, and importing the historical data to be migrated into the historical data migration tool;
the historical data export unit is used for exporting the historical data to be migrated to the target system in response to a data export instruction so that a user can keep a data interface of the system to be migrated on a data interface checked by the target system;
The compatibility adaptation of the high-version operating system, the browser and the target system for the page elements for data display in the migration module comprises the following steps:
determining a main page corresponding to the data record in the migration module, judging whether the main page has a sub page, and performing compatibility adaptation on sub page elements of the sub page first and then performing compatibility adaptation on main page elements of the main page when the main page has the sub page;
performing the compatibility adaptation on the main page element when the sub page does not exist in the main page;
the compatibility adaptation of the main page element or the sub page element includes:
based on a preset compatibility adaptation rule, carrying out compatibility adaptation on the target page element; the target page element is the main page element or the sub page element; the preset compatibility adaptation rule comprises that aiming at the target page element, a page style is modified into a compatible high-version browser, a page frame style is modified into a preset page frame compatible style, element centering codes are added aiming at preset element content, access paths of css files and js files in the target page are respectively modified into preset access modes, a storage mode of gif files is modified into a preset storage mode, a page modification button of the target page is shielded, an accessory opening mode is modified into a preset opening mode, and calling interface programs are reserved before and after loading the target page; the target page is a page corresponding to the target page element.
7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the historical data migration method of any one of claims 1 to 5 when the program is executed by the processor.
8. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the historical data migration method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311571005.5A CN117271482B (en) | 2023-11-23 | 2023-11-23 | Historical data migration method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311571005.5A CN117271482B (en) | 2023-11-23 | 2023-11-23 | Historical data migration method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117271482A CN117271482A (en) | 2023-12-22 |
| CN117271482B true CN117271482B (en) | 2024-02-27 |
Family
ID=89201299
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311571005.5A Active CN117271482B (en) | 2023-11-23 | 2023-11-23 | Historical data migration method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117271482B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108614869A (en) * | 2018-04-17 | 2018-10-02 | 国家电网公司 | A kind of method of historical data in different automation system platform synchronous migrations |
| CN109933416A (en) * | 2019-02-01 | 2019-06-25 | 广东安可云科技有限公司 | A kind of moving method, device and storage medium using data |
| CN110737645A (en) * | 2019-10-15 | 2020-01-31 | 深圳金蝶账无忧网络科技有限公司 | data migration method between different systems, data migration system and related equipment |
| CN114047949A (en) * | 2021-10-27 | 2022-02-15 | 中国国家铁路集团有限公司 | Application system domestic platform migration adaptation method |
| CN114610384A (en) * | 2020-12-08 | 2022-06-10 | 腾讯科技（深圳）有限公司 | Application migration method, device, equipment and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180191825A1 (en) * | 2016-12-30 | 2018-07-05 | Cerner Innovation, Inc. | Migrating, editing, and creating content between different collaboration systems |
| US11556699B2 (en) * | 2019-02-04 | 2023-01-17 | Citrix Systems, Inc. | Data migration across SaaS applications |
-
2023
- 2023-11-23 CN CN202311571005.5A patent/CN117271482B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108614869A (en) * | 2018-04-17 | 2018-10-02 | 国家电网公司 | A kind of method of historical data in different automation system platform synchronous migrations |
| CN109933416A (en) * | 2019-02-01 | 2019-06-25 | 广东安可云科技有限公司 | A kind of moving method, device and storage medium using data |
| CN110737645A (en) * | 2019-10-15 | 2020-01-31 | 深圳金蝶账无忧网络科技有限公司 | data migration method between different systems, data migration system and related equipment |
| CN114610384A (en) * | 2020-12-08 | 2022-06-10 | 腾讯科技（深圳）有限公司 | Application migration method, device, equipment and storage medium |
| CN114047949A (en) * | 2021-10-27 | 2022-02-15 | 中国国家铁路集团有限公司 | Application system domestic platform migration adaptation method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117271482A (en) | 2023-12-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9892263B2 (en) | System, method and apparatus to visually configure an analysis of a program | |
| US11507655B1 (en) | Automatic and predictive source code generation | |
| CN110659569B (en) | Electronic signature method, device, storage medium and electronic equipment | |
| CN111767095A (en) | Micro-service generation method and device, terminal equipment and storage medium | |
| US11620570B2 (en) | Self-learning ontology-based cognitive assignment engine | |
| US11526340B2 (en) | Providing context-based application suggestions | |
| US10599755B2 (en) | System and method for automatically inserting correct escaping functions for field references in a multi-tenant computing environment | |
| US11841972B2 (en) | System and method to safeguarding sensitive information in cobrowsing session | |
| CN111767109A (en) | H5 page display method and device based on terminal application and readable storage medium | |
| CN110519155B (en) | Information processing method and system | |
| US20210397646A1 (en) | Method, electronic device, and storage medium for processing visual data of deep model | |
| US11586696B2 (en) | Enhanced web browsing | |
| CN117271482B (en) | Historical data migration method and device, electronic equipment and storage medium | |
| CN115858049A (en) | RPA flow componentization arrangement method, device, equipment and medium | |
| EP3425510B1 (en) | Automated testing method for application program user interface, electronic device, system, and storage medium | |
| CN115145674A (en) | Page jump method, device, equipment and medium based on dynamic anchor point | |
| CN113590564A (en) | Data storage method and device, electronic equipment and storage medium | |
| WO2021234019A1 (en) | Database management methods and associated apparatus | |
| CN111679862A (en) | Cloud host shutdown method and device, electronic equipment and medium | |
| US10572281B1 (en) | Bi-directional notification service | |
| CN117407430B (en) | Data query method, device, computer equipment and storage medium | |
| US20230315493A1 (en) | Inter-platform control interface | |
| CN105378605B (en) | Change the input to application delivering | |
| US20160062974A1 (en) | Recording reasons for metadata changes | |
| US20120296992A1 (en) | Instant messaging validation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |