Does directly adding relative-position-embedding and absolute-position-embedding provide the same length extrapolation advantages for LLMs by RoPE?

15 March 2025 0 6K Report

Does directly adding relative-position-embedding and absolute-position-embedding provide the same length extrapolation advantages for LLMs by RoPE (Rotary Position Embedding)?

Badges
Science topic

More Tong Guo's questions See All

"A Markov-like Model for Patient Progression"?

A Markov-like Model for Patient Progression" Markov Chain Monte Carlo (MCMC) Markov Chain Monte Carlo (MCMC) is a powerful computational technique used to draw samples from a probability...

05 August 2024 10,079 0 View

La animación digital en plataformas digitales?

Hoy la animación se utiliza como una tecnología multimedia con gran potencial educativo, que va mucho más allá de sólo crear figuras, ya que puede promover una mejor comprensión en...

01 August 2024 7,186 0 View

GSH estimation assay: What is the right choice of standard?

Hi there, My question is: What standard curves should be used while estimating Tot GSH and GSSG by kinetic method using GR enzyme mediated recyling with DTNB chromophore? Actually I am following...

01 August 2024 8,217 1 View

How to do pca analysis of c-alpha atom of the protein?

i m interested in pca analysis of c-alpha atoms in gromacs for that i used the following gmx_mpi covar -s mdca.tpr -f mdca.xtc -o eigenvalca.xvg -v eigenvecca.trr -av average.pdb -n index.ndx but...

30 July 2024 1,607 1 View

What exactly is RAG-LLM doing? Isn’t it data engineering?

What exactly is Retrieval Augmented Generation for Large Language Model doing? Isn’t it data engineering?

30 July 2024 7,376 3 View

After a lot of feature engineering for CTR modeling, it feels like it's basically the end of iteration? I mean, it's not cost-effective to keep doing?

After a lot of feature engineering for click-through rate modeling, it feels like it's basically the end of iteration? I mean, it's not cost-effective to keep doing it?

29 July 2024 4,955 0 View

How to estimate sample size for GWAS of continuous and discrete traits? What are the pre-requisites?

Genome-wide association study (GWAS) Continuous traits: eg. Height Discrete traits: eg. Eye color

28 July 2024 286 0 View

All math can be explained by iterator of code?

all math can be traversed by code? all math can be translate to code?

26 July 2024 9,530 0 View

HEC 1A & HEC1B Cell Lines?

Hi, Kindly guide me that how many cells of HEC1A & HEC1B Cell lines should I seed for Wound healing assay and which plate type is recommended 6, 12 & 24?. Articles suggested mainly 24...

20 July 2024 4,143 2 View

Why electrical charge on the moving plate increase?

Hi, everyone This figure depicts a simulation of an electrostatic energy harvesting system in COMSOL Multiphysics software. My question is regarding the relationship between the changes in...

19 July 2024 4,694 4 View

How can I prepare virus for a TEM or SEM imaging?

I have virus (viral hemorrhagic septicemia virus) in suspension and the experiment will not involve cells. What level of TCID50 is preferred?

11 August 2024 3,115 1 View

Handling Missing Data and Building a Predictive Model with Incomplete Information ?

I am developing a predictive model for a water supply network that involves 20 influencing points. However, I only have historical data for 10 out of these 20 points. I would like to know how to...

10 August 2024 4,005 2 View

Is it possible to use the Fused Deposition Modeling (FDM) to additively manufacture interconnected porous structure generation of >100-200 micrometer?

Usually, additive manufacturing techniques like SEBM, SLS, and SLM are used for interconnected porous lattice structure generation with sizes of >100–200 micrometers. Can the Fused Deposition...

09 August 2024 7,892 0 View

How to define an anisotropic material with asymmetric elastic compliance/stiffness matrix in ANSYS APDL?

I need to model an anisotropic material in which the Poisson's ratio ν_12 ≠ ν_21 and so on. Therefore, the elastic compliance matrix wouldn't be a symmetric one. In ANSYS APDL, for TB,ANEL...

09 August 2024 5,048 2 View

How can I apply boundary conditions in an orthotropic steel deck numerical model using ABAQUS software?

I am trying to simulate vehicular loading on an orthotopic steel deck bridge section in ABAQUS software. The red arrow mark in the attached figure indicates the direction in which the vehicle will...

08 August 2024 719 0 View

Can you suggest reliable sources defining "3D mesh" and "3D city models"?

Dear fellow researchers, I am currently working on a paper where I need to provide a reliable reference that defines and distinguishes between 3D mesh models and 3D city models. Although I am...

06 August 2024 9,986 2 View

Please explain how the plastic input value should be considered from the true stress-strain curve for the bilinear elastoplastic material model ?

I am working on Abaqus/Explicit(Quasistatic ) for the deformation of the auxetic structure model. Please explain how the plastic input value should be considered from the true stress-strain curve...

05 August 2024 454 3 View

What are the shear and normal stiffness values of an LLDPE liner in 3D numerical modeling of a stockpile?

I am seeking experimental or applicable data for the liner (LLDPE) interface in FLAC3D numerical modeling of a large stockpile. Could you please recommend suitable references? The preferred data...

05 August 2024 3,665 0 View

Is it necessary to covary exogenous constructs in a structural model?

I am working on a SEM model where i have 7 latent variables (6 exogenous and 1 endogenous). In AMOS when I co-vary the exogenous constructs, only 2 paths are coming significant out of 6. But when...

03 August 2024 6,028 4 View

Broca’s area must be intact for the learning of new movement sequences?

When the eyes of a person are damaged this causes complete blindness. Likewise, when Wernicke’s and Broca’s areas of neocortex are damaged this causes complete aphasia, losing the ability to...

01 August 2024 6,744 2 View